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Why GAO Did This Study 

Hurricane Katrina was the most 
destructive disaster in our nation's 
history and it highlighted gaps in 
preparedness for a catastrophic 
disaster. The Federal Emergency 
Management Agency (FEMA), a 
component within the Department 
of Homeland Security (DHS), is the 
lead federal agency responsible for 
developing a national preparedness 
system. The system includes 
policies and plans as well as 
exercises and assessments of 
capabilities across many public and 
private entities. GAO was asked to 
assess the extent to which FEMA 
has (1) developed policies and 
plans that define roles and 
responsibilities; (2) implemented 
the National Exercise Program, a 
key tool for examining 
preparedness; (3) developed a 
national capabilities assessment; 
and (4) developed a strategic plan 
that integrates these elements of 
the preparedness system. GAO 
analyzed program documents, such 
as after-action reports, and visited 
six states located in disaster 
regions. While the results of these 
visits are not generalizable, they 
show how select states carry out 
their efforts. 



What GAO Recommends 



GAO recommends that FEMA 
improve national preparedness by, 
among other things, establishing a 
program management plan, better 
ensuring exercises follow program 
guidance, enhancing its project 
management plan for assessing 
capabilities, and developing a 
strategic plan that integrates 
system elements. DHS concurred 
with our recommendations. 

To view the full product, including the scope 
and methodology, click on GAO-09-369. 
For more information, contact William O. 
Jenkins, Jr., at (202) 512-8757 or 
jenkinswo @ gao.gov. 



What GAO Found 

While most policies (41 of 50) that define roles and responsibilities have been 
completed, such as the National Response Framework, 68 percent (49 of 72) 
of the plans to implement these policies, including several for catastrophic 
incidents, are not yet complete. As a result, the roles and responsibilities of 
key officials involved in responding to a catastrophe have not been fully 
defined and, thus, cannot be tested in exercises. The lack of clarity in 
response roles and responsibilities among the diverse set of responders 
contributed to the disjointed response to Hurricane Katrina and highlighted 
the need for clear, integrated disaster preparedness and response policies and 
plans. Although best practices for program management call for a plan that 
includes key tasks and their target completion dates, FEMA does not have 
such a plan. With such a plan, FEMA would be better positioned to ensure 
that the policies and plans are completed and integrated with each other as 
intended as well as with other elements of the preparedness system. 

Since 2007, FEMA has taken actions to implement the National Exercise 
Program at the federal and state levels by developing, among other things, 
program guidance and systems to track corrective actions; however, FEMA 
faces challenges in ensuring that the exercises are carried out consistent with 
program guidance. For example, the Homeland Security Council (an 
interagency entity responsible for coordinating homeland security policy) and 
state participants did not systematically track whether corrective actions had 
been taken to address deficiencies identified by exercises as called for by 
program guidance. As a result, FEMA lacks reasonable assurance that entities 
have taken actions aimed at improving preparedness. 

FEMA has made progress in developing a system for assessing national 
preparedness capabilities by, among other things, establishing reporting 
guidance for state preparedness, but it faces challenges in completing the 
system and required reports to assess preparedness. While FEMA has 
developed a project management plan for the new system, the plan does not 
fully identify milestones and program risks for developing quantifiable metrics 
necessary for measuring preparedness capabilities. A more complete project 
plan that identifies milestones and program risks would provide FEMA with 
greater assurance that it can produce a system to assess capabilities and 
inform decisions related to improving national preparedness. 

FEMA's strategic plan for fiscal years 2008-2013 recognizes that each of its 
components need to develop its own strategic plans that integrate the 
elements of national preparedness. FEMA's National Preparedness 
Directorate has yet to develop its strategic plan, but instead plans to use a 
draft annual operating plan to guide its efforts. This plan does not include all 
elements of a strategic plan, such as how the directorate will integrate the 
various elements of the system over time to improve national preparedness. 
Having a strategic plan would provide FEMA with a roadmap for addressing 
the complex task of guiding and building a national preparedness system. 
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United States Government Accountability Office 
Washington, DC 20548 



April 30, 2009 

The Honorable Joseph I. Lieberman 
Chairman 

The Honorable Susan M. Collins 
Ranking Member 

Committee on Homeland Security and Governmental Affairs 
United States Senate 

The Honorable Bennie G. Thompson 
Chairman 

Committee on Homeland Security 
House of Representatives 

The attacks of 9/11 and Hurricane Katrina were, respectively, the most 
destructive terrorist and natural disasters in our nation's history and 
highlighted gaps in the nation's readiness to respond effectively to large- 
scale catastrophes. Among the gaps were a lack of clarity in roles and 
responsibilities of federal, state, and local officials; limitations in assisting 
those with special needs, such as residents in nursing homes; and 
logistical problems in providing food, water, and other goods to those in 
areas affected by the disaster. To strengthen the nation's preparedness for 
such incidents, in December 2003 the President issued a directive that 
called on the Secretary of Homeland Secretary to enhance capabilities of 
federal, state, and local entities. 1 In October 2006, the Post-Katrina 
Emergency Management Reform Act (Post-Katrina Act) charged the 
Federal Emergency Management Agency (FEMA) — a major component of 
the Department of Homeland Security (DHS) — with responsibility for 
leading the nation in developing a national preparedness system. 2 

The size and complexity of the nation's preparedness activities and the 
number of organizations involved make developing a national 
preparedness system a difficult task. To lead national preparedness 
efforts effectively, FEMA is to coordinate with a wide range of federal 
departments and agencies, such as the Departments of Defense, Health 



homeland Security Presidential Directive 8 — National Preparedness (Dec. 17, 2003). 

2 The Post-Katrina Act was enacted as Title VI of the Department of Homeland Security 
Appropriations Act, 2007, Pub. L. No. 109-295, 120 Stat. 1355 (2006). The Post-Katrina Act 
is codified in numerous sections of the U.S. Code, which we will cite in this report. 
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and Human Services, and Justice; 50 states, the District of Columbia, and 5 
territories; city and county governments; a wide range of nonprofit 
organizations; and private entities, such as debris removal contractors. 
Each of these entities has many stakeholders with whom FEMA is to 
coordinate and partner, such as governors and mayors, state and local 
emergency managers, and first responders. 

The national preparedness system, as shown below in figure 1, is a 
continuous cycle of (1) establishing policy and doctrine, (2) planning and 
allocating resources, (3) conducting training and exercises to gather 
lessons learned, and (4) assessing and reporting on the training and 
exercises to evaluate preparedness, including identifying any gaps in 
capabilities. The results of these assessments and reports are then used to 
inform decision makers on what improvements are needed in policies and 
plans and how to target finite resources to improve preparedness for 
disasters. 
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Figure 1 : Elements of the National Preparedness System 




Source: Adapted from FEMA website: http://www.dhs.gov/gov/prepesp/publications. 

"Policy and Doctrine involves ongoing management and maintenance of national policy and doctrine 
for operations and preparedness. For purposes of this report, policy and doctrine includes certain 
laws and executive orders that are relevant to national preparedness, as well as the policy documents 
that derive from them, such as the National Preparedness Guidelines. Issued by DHS in September 
2007, the National Preparedness Guidelines involve several key elements, including the National 
Preparedness Vision, which provides a concise statement of the core preparedness of the nation. 

"Planning and Resource Allocation involves application of common planning processes to identify 
requirements, allocate resources, and build and maintain coordinated capabilities that are prioritized 
based upon risk. The application of these planning processes results in the development of strategic, 
conceptual and operational plans, as well as scenario-specific plans. 

"Training, Exercises, and Lessons Learned involves delivery of training and exercises and 
performance evaluation to identify lessons learned and share effective practices. 
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"Assessment and Reporting involves assessments based on established readiness metrics and 
reporting on progress and effectiveness of preparedness efforts. 

Since 2005, we have issued several reports on the progress, challenges, 
and weaknesses that exist in carrying out various preparedness programs, 
including exercises and assessment efforts. 3 Given the critical role that 
FEMA plays in homeland security and emergency response, you asked us 
to review the progress that FEMA has made to develop key elements of 
the national preparedness system. 4 As a result, this report examines 
several key parts of the system in more detail: policy and doctrine; 
planning, exercises and lessons learned; and assessments and reporting. 
Policies broadly define roles and responsibilities, while plans 
operationalize policies by providing more specific information on roles 
and responsibilities and associated tasks. 5 FEMA's National Exercise 
Program carries out exercises which are a key tool for testing and 
evaluating preparedness. Assessments include not only assessments of 
individual exercises and other preparedness efforts, but of capabilities to 
determine what gaps, if any, exist in capability levels. FEMA's ability to 
establish the national preparedness system depends, in part, on the 
effectiveness with which it integrates the elements of the system and 
coordinates with stakeholders, including federal, state, and local agencies, 
involved in national preparedness. 6 



3 See GAO, Homeland Security: DHS' Efforts to Enhance First Responders ' All-Hazard 
Capabilities Continue to Evolve, GAO-05-652 (Washington D.C.: Jul. 11, 2005); 
Catastrophic Disasters: Enhanced Leadership, Capabilities, and Accountability 
Controls Will Improve the Effectiveness of the Nation's Preparedness, Response, and 
Recovery System, GAO-06-618 (Washington D.C.: Sept. 6, 2006); Disaster Preparedness: 
Better Planning Would Improve OSHA's Efforts to Protect Workers' Safety and Health in 
Disasters, GAO-07-193 (Washington D.C.: Mar. 28, 2007); Continuity of Operations: 
Selected Agencies Tested Various Capabilities during 2006 Governmentwide Exercises, 
GAO-08-185 (Washington D.C.: Nov. 19, 2007). 

4 Because FEMA may have collaborated with other DHS components in carrying out its 
responsibilities, our report may refer to both DHS and FEMA where appropriate. 

°Since FEMA uses the terms "policy" and "doctrine" interchangeably, we will refer to 
"policy" or "policies" when discussing the "Policy and Doctrine" element of the National 
Preparedness System. For the purposes of this report, plans developed using planning 
processes include strategic, conceptual, operational, and scenario-specific plans. 

6 We did not examine resource allocation and training. 
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Specifically, this report addresses the extent to which FEMA has: 

• developed policies and plans to define roles and responsibilities and 
planning processes for national preparedness; 

• taken actions since 2007 to implement the National Exercise Program 
and to track corrective actions at the federal and state levels and what 
challenges remain; 

• made progress in conducting a nationwide capabilities-based 
assessment, including developing required preparedness reports, and 
what issues, if any, it faces in completing the system; and 

• developed a strategic plan for implementing the national preparedness 
system. 

To address the questions above, we analyzed information and data on 
FEMA's policies and plans for preparedness, implementation of the 
National Exercise Program, its approach for developing a comprehensive 
system for assessing nationwide capabilities, and its strategy for 
integrating elements of the preparedness system. We analyzed key 
legislation such as the Post-Katrina Act and the Implementing 
Recommendations of the 9/11 Commission Act of 2007 ("9/11 Act") as well 
as presidential directives related to preparedness efforts. 7 We interviewed 
FEMA officials responsible for preparedness programs to learn more 
about the actions they had taken and planned to take related to 
preparedness efforts. Specifically, 

• To analyze the extent to which FEMA has developed policies and plans 
to define roles and responsibilities and planning processes for national 
preparedness, we reviewed the policies and plans that form the basis of 
the preparedness system. These policies and plans include, among 
others, the National Response Framework (NRF) and National 
Preparedness Guidelines, as well as the national integrated planning 
system and preliminary versions of related guidance to develop and 



7 Pub. L. No. 110-53, 121 Stat. 266 (2007). 
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integrate plans across federal, state, tribal, and local governments. 8,9 
We summarized lessons learned from after-action reports for federal 
exercises from 2005 through 2008 to determine whether the exercises 
revealed unclear or conflicting roles and responsibilities between 
federal departments and agencies, and whether additional policies or 
plans were identified as needed corrective actions. We also compared 
FEMA's approach for ensuring policies and plans are completed with 
best practices for program management established by the Project 
Management Institute. 10 
• To assess the extent to which FEMA has taken actions since 2007 to 
implement a National Exercise Program and to track corrective 
actions, we evaluated key program documents, such as the National 
Exercise Program Implementation Plan (hereafter referred to as the 
implementation plan); the Homeland Security Exercise and Evaluation 
Program (HSEEP) — FEMA's guidelines for carrying out exercises; and 
data on the program's performance measures. We evaluated actions 
taken by FEMA since 2007 because the implementation plan was issued 
in April 2007. We examined after-action reports for Principal Level 
Exercises — which involve senior federal officials, such as deputy 
secretaries of departments or agencies — that were produced from April 
2007 through August 2008 to determine whether a system was in place 
to ensure that lessons learned were incorporated into national 



Issued by DHS in January 2008, the NRF is the doctrine that guides how federal, state, 
local, and tribal governments, along with nongovernmental and private sector entities, will 
collectively respond to and recover from all hazards, including catastrophic disasters such 
as Hurricane Katrina. The NRF consists of a core base document that broadly describes 
the roles and responsibilities of key officials. In addition, roles and responsibilities for 
response activities are further defined by 15 NRF Emergency Support Function (ESF) 
Annexes, while 8 additional NRF Support Annexes describe the execution of common 
incident management functional processes and administrative requirements. The NRF also 
includes 4 Partner Guides, which are to provide ready references describing key roles and 
actions for local, tribal, state, federal, private-sector, and nongovernmental response 
partners. Finally, the NRF contains 7 NRF Incident Annexes and and 2 NRF Incident 
Annex Supplements, which are to address aspects of how the United States responds to 
broad incident types. 

9 Titled "National Planning," Homeland Security Presidential Directive 8 Annex 1 is intended 
to enhance the preparedness of the United States by formally establishing, developing, and 
maintaining a standard and comprehensive approach to national planning. It calls for, 
among other things, a standardized federal planning process (the Integrated Planning 
System); a family of strategic guidance statements, strategic plans, concepts of operations, 
operations plans, and, as appropriate, tactical plans for responding to each National 
Planning Scenario; and a system for integrating plans among all levels of government. 

10 Project Management Institute, The Standard for Program Management, 2 nd ed. (Newton 
Square, Pa.: 2006). 
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preparedness. 11 We also interviewed Homeland Security Council 
(HSC) staff and the Associate Counsel to the President on the role and 
responsibilities of the council in designing, executing, and following up 
on corrective actions resulting from Principal Level Exercises because 
these staff were responsible for summarizing lessons learned and 
corrective actions for such exercises. For information that would 
provide a broader perspective on FEMA's efforts, we examined several 
FEMA databases, including the FEMA Secure Portal— the FEMA 
repository of after-action reports; the National Exercise Schedule 
(NEXS) system — a scheduling system for all exercises; and the 
Corrective Action Program (CAP) System — which was designed for 
tracking capability improvement plans entered by federal, state, and 
local exercise participants. We assessed the reliability of these 
databases by checking them against documents, such as after-action 
reports produced by states, and by interviewing staff responsible for 
the data. We concluded the data in the FEMA Secure Portal, NEXS 
system, and CAP system were not reliable for use in this report. We 
discuss the reliability of these data later in this report. Finally, we 
compared FEMA's policies and procedures for implementing the 
National Exercise Program with criteria in GAO's standards for internal 
control in the federal government. 12 
• To determine the extent to which FEMA has made progress and what 
issues, if any, remain in conducting a nationwide capabilities-based 
assessment and developing required preparedness reports, we analyzed 
information pertinent to FEMA's assessment approach, such as State 
Preparedness Reports and the Federal Preparedness Report. 13 We also 
examined FEMA's plans for implementing the comprehensive 
assessment system. We interviewed FEMA staff responsible for 



"The National Exercise Program involves, among other things, tier I exercises that include 
one national level exercise and up to four Principal Level Exercises annually. One of the 
four annual Principal Level Exercises serves as a preparatory event for the annual national 
level exercise. 

12 GAO Standards for Internal Control in the Federal Government, GAO/AIMD-00-21.3.1 
(Washington, D.C.: November 1999). These standards, issued pursuant to the requirements 
of the Federal Managers' Financial Integrity Act of 1982 (FMFIA), provide the overall 
framework for establishing and maintaining internal control in the federal government. 
Also pursuant to FMFIA, the Office of Management and Budget (OMB) issued Circular A- 
123, revised December 21, 2004, to provide the specific requirements for assessing the 
reporting on internal controls. Internal control standards and the definition of internal 
control in OMB Circular A-123 are based on GAO's Standards for Internal Control in the 
Federal Government. 

13 Department of Homeland Security, Federal Emergency Management Agency, The Federal 
Preparedness Report (Washington, D.C.: January 2009). 
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developing FEMA's assessment system and performance objectives for 
measuring capabilities. Finally, we reviewed our prior reports on 
FEMA's preparedness programs. 
• To determine the extent to which FEMA has developed a strategic plan 
that integrates the national preparedness system, we analyzed key 
FEMA documents including the agencywide Strategic Plan for Fiscal 
Years 2008 through 2013, the Grant Programs Directorate Strategic 
Plan, and the draft annual National Preparedness Directorate Operating 
Plan. We interviewed FEMA National Preparedness Directorate staff 
on strategic planning and policy and procedures for the national 
preparedness system. We also compared FEMA's current approach for 
developing a national preparedness system with the desirable 
characteristics of effective national strategies we identified in February 
2004. 14 

To supplement our analyses of FEMA's preparedness activities, we visited 
six states — California, Georgia, Illinois, New York, Texas, and Washington. 
While we cannot generalize our work from these visits to all states, we 
chose these locations for their geographic diversity and to provide 
examples of the way in which states carry out their exercise and 
preparedness programs. In selecting these states, we considered factors 
such as states' participation in national level exercises; states located in 
hurricane-prone regions; and states with varying percentages of homeland 
security grant funding planned to support exercises. At each location, we 
interviewed staff in FEMA's regional offices responsible for implementing 
regional requirements of the National Exercise Program and regional 
preparedness activities. We also interviewed officials in each state on 
their progress and challenges in carrying out preparedness activities, 
including exercises and assessments of capability. In addition, we 
evaluated after-action reports produced by each of the six states. 

We conducted this performance audit from January 2008 through April 
2009 in accordance with generally accepted government auditing 
standards. Those standards require that we plan and perform the audit to 
obtain sufficient, appropriate evidence to provide a reasonable basis for 
our findings and conclusions based on our audit objectives. We believe 
that the evidence obtained provides a reasonable basis for our findings 
and conclusions based on our audit objectives. Appendix I contains more 
details on our scope and methodology. 



GAO, Combating Terrorism: Evaluation of Selected Characteristics in National 
Strategies Related to Terrorism, GAO-04-408T (Washington, D.C.: Feb. 3, 2004). 
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Results in Brief DHS, FEMA, and other federal entities have completed most of the key 

policies (42 of 50), such as the base NRF, that define roles and 
responsibilities for emergency preparedness and response and the 
planning processes for developing emergency plans. However, about 68 
percent (49 of 72) of the associated plans to implement these policies, 
such as certain operational plans for terrorism response, are incomplete. 
In 2008, DHS published the base NRF, which broadly describes national 
response doctrine and the roles and responsibilities of officials involved in 
response efforts, and certain related annexes, but DHS has not yet 
completed other components of the NRF, such as the four response 
partner guides that are to provide stakeholder-specific references 
describing key roles for federal and nonfederal leaders. Real-world events 
have shown that incomplete policies and plans may adversely impact 
response efforts. For example, the White House and GAO concluded that 
incomplete policies and plans contributed to the lack of clarity in 
leadership roles and responsibilities in the response to Hurricane Katrina. 
This problem resulted in disjointed efforts by emergency responders that 
may have caused increased losses of life and property. Best practices for 
program management state that a key step in managing a program involves 
developing and utilizing a program management plan, which defines how a 
program will be planned, executed, monitored, and controlled. 15 FEMA 
has not established a program management plan, in coordination with 
other federal departments and agencies, to help ensure the development 
and integration of outstanding policies and plans. However, based in part 
on our work, in February 2009 FEMA officials acknowledged that a 
program management plan should be established. Further, until the 
national preparedness system includes a complete and integrated set of 
policies and plans that lay out roles and responsibilities and planning 
processes, FEMA's ability to prepare officials responsible for responding 
to natural and man-made disasters will be limited. By establishing a 
program management plan, FEMA would be better positioned to ensure 
that the policies and plans are developed and integrated with each other 
and the national preparedness system as envisioned by law and 
presidential directive. 

Since 2007, FEMA has taken actions to implement the National Exercise 
Program and track corrective actions, including developing an 
implementation plan which establishes requirements for exercises that 



°Project Management Institute, The Standard for Program Management, 2° d ed. (Newton 
Square, Pa.: 2006). 
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involve participation among federal entities; however, FEMA faces 
challenges in meeting statutory requirements for tracking corrective 
actions and for measuring the effectiveness and progress of the National 
Exercise Program. First, although the Post-Katrina Act requires FEMA to 
track corrective actions from exercises, FEMA has not been able to do so 
in an effective manner because neither HSC — a federal entity responsible 
for coordinating interagency homeland security policy — nor all the states 
have followed up on corrective actions in a way that ensured 
improvements in national preparedness. For example, the after-action 
reports we reviewed from HSC did not include information on the time 
frames or entities responsible for completing corrective actions and 
whether federal entities carried out corrective actions, as called for by 
program guidelines. Without such information, federal agencies cannot be 
held accountable for implementing corrective actions that aim to improve 
preparedness. Moreover, this problem is not new. More than 3 years ago, 
a February 2006 White House report on Hurricane Katrina stated that DHS 
should ensure that all federal and state entities carry out remedial actions 
in a timely way. This problem occurred, in part, because FEMA does not 
have procedures in place that detail how it will work with HSC to ensure 
that corrective actions are tracked and implemented. In addition to 
weaknesses at the federal level, five of the six states we visited did not 
have a program for tracking corrective actions, as required by program 
requirements. This weakness occurred because FEMA lacked procedures 
for effectively monitoring compliance by states with National Exercise 
Program requirements. Weaknesses in procedures for tracking corrective 
actions has led, in turn, to a second challenge faced by FEMA, which 
involves the reliability of several databases used by FEMA to measure the 
progress of the National Exercise Program. For example, HSC did not use 
a FEMA database that tracks the status of corrective actions or otherwise 
provide FEMA with necessary corrective action information. Program 
guidelines encourage the use of this database, but do not require that 
federal agencies use it. Without complete information on the status of 
corrective actions, FEMA cannot effectively assess the progress and 
performance of the National Exercise Program in making improvements in 
national preparedness. 

FEMA has made progress in developing and implementing a system for 
assessing national preparedness capabilities, but faces methodological and 
coordination challenges in completing the system and issuing required 
reports on national preparedness. The Post-Katrina Act requires that 
FEMA establish a comprehensive system to assess the nation's prevention 
capabilities and overall preparedness. Such a system should assess, 
among other things, (1) the nation's preparedness capability levels against 
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levels defined by desired, or "target" capabilities needed to prevent, 
respond to, and recover from natural and man-made disasters and (2) 
resources needed to meet the desired target capabilities. FEMA has 
established reporting guidance for state preparedness and has created a 
program office to develop and implement an assessment approach that 
considers past efforts and integrates its ongoing efforts related to 
measuring the results of federal grants and assessing gaps in disaster 
response capabilities. However, FEMA faces challenges in developing and 
completing this approach. As noted in the January 2009 Federal 
Preparedness Report, efforts to assess capabilities and make needed 
improvements are the least mature elements of the national preparedness 
system because these efforts are composed of a wide range of systems and 
approaches with varying levels of integration. FEMA faces 
methodological challenges that include deciding how information and data 
from different sources will be used to inform the system and developing an 
approach for coordinating with federal, state, and local stakeholders in 
developing and implementing the system and reporting on its results. 
Moreover, FEMA has faced similar challenges in three previous attempts 
to assess capabilities since at least 1997. For example, from 2006 through 
2008, FEMA spent $15 million on a Web-based system to identify capability 
data among states. However, FEMA discontinued the effort, in part 
because the data produced were not meaningful. Best practices for 
project management call for milestone dates; an assessment of risks, such 
as the methodological challenges described above, in carrying out a 
project successfully; and identification of ways to mitigate such risks. 
While FEMA has developed a project management plan for completing the 
comprehensive assessment system, this plan does not fully identify 
program elements to complete the system, such as milestones and 
program risks for developing quantifiable metrics necessary for measuring 
target capability levels. FEMA's ability to establish a system that produces 
meaningful data will be enhanced by detailed plans that include milestone 
dates and program risks. 

FEMA's agencywide strategic plan for fiscal years 2008 through 2013 calls 
on its components to develop individual strategic plans; however, the 
National Preparedness Directorate has yet to complete a plan for the 
national preparedness system. 16 Instead, according to the former director 
of the Preparedness Directorate, the directorate uses the Post-Katrina Act 



16 The Post-Katrina Act also calls on FEMA to develop and coordinate the implementation of 
a strategy for preparedness. 6 U.S.C. § 313(b)(2)(H). 
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and an annual draft operating plan to guide its strategic approach. The 
Post-Katrina Act and the draft operating plan contain some, but not all 
elements of a strategic plan. For example, they do not describe how the 
directorate will (1) measure its progress in developing the preparedness 
system, (2) address risks as they relate to preparedness activities, (3) 
coordinate with its stakeholders in developing and carrying out the 
various elements of the National Preparedness System, and (4) implement 
and integrate the system. The complexity, difficulty, and importance of 
the preparedness system underscore the importance of a strategic plan in 
the development, implementation, and integration of the system. In 
February 2004, we identified six characteristics of effective strategies that 
strengthen program success that could be applied to the National 
Preparedness System. Those characteristics are: (1) a statement of 
purpose, scope, and methodology; (2) problem definition and risk 
assessment; (3) goals, subordinate objectives, activities, and performance 
measures; (4) resources, investments, and risk management; (5) 
organizational roles, responsibilities, and coordination; and (6) integration 
and implementation. Developing a strategic plan for the national 
preparedness system that outlines clear, measurable objectives and goals, 
including who is responsible for achieving them, would assist FEMA in 
improving accountability of organizational entities involved in 
preparedness efforts and enhance the likelihood of its success in using 
program results to improve policies, plans, exercises, and capabilities 
related to national preparedness. 

We make a number of recommendations to the Administrator of the 
Federal Emergency Management Agency to help address weaknesses in 
efforts to develop policies and plans that define roles and responsibilities, 
carry out the National Exercise Program, develop the comprehensive 
assessment system, and implement a strategic approach for the national 
preparedness system. These recommendations cover such matters as 
developing a program management plan to help ensure the completion of 
policies and plans that define roles and responsibilities, developing 
procedures that detail how FEMA will work with HSC to help ensure that 
the council documents and tracks corrective actions resulting from 
exercises, revising FEMA's procedures for monitoring compliance by 
states with National Exercise Program requirements, developing a more 
detailed project management plan for implementing the comprehensive 
assessment system, and developing a strategic plan to help ensure the 
implementation of the national preparedness system. 

In commenting on a draft of this report, DHS said it generally agreed with 
our recommendations. However, DHS expressed concern that the report 
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suggests that DHS/FEMA should hold other federal agencies and 
departments or state, local or tribal governments accountable for 
compliance with program requirements, while also recognizing that FEMA 
did not always have the explicit authority to compel compliance. The 
Post-Katrina Act designates FEMA as the federal leader and coordinator 
for developing and implementing the national preparedness system. We 
recognize that FEMA's authority is generally to coordinate, guide, and 
support, rather than direct, and that collaboration is an essential element 
of FEMA's efforts. At the same time, we believe that FEMA's expanded 
leadership role under the Post-Katrina Act provides it with opportunities 
for and a responsibility to further develop its relationships with national 
preparedness stakeholders at the local, state and federal levels and to 
instill a shared sense of responsibility and accountability on the part of all 
stakeholders for the successful development and implementation of the 
national preparedness system. Several of our recommendations aim to 
enhance such collaboration and cooperation. 



BctckSrOlind ^ Hurricane Katrina dramatically illustrated the adverse 

" consequences that can occur when the nation is unprepared to respond 

effectively to a catastrophic disaster. Emergency preparedness 
strengthens the nation's ability to prevent, protect, respond to, and recover 
from a natural disaster, terrorist attack, or other man-made disaster. It has 
received widespread attention and support from Congress, the President, 
and the Secretary of Homeland Security as manifested by legislation, 
presidential directives, the development of DHS policy documents, and 
grants to state and local governments. The lessons learned from the 
terrorist attacks of 9/11 and Hurricane Katrina focused attention on the 
need for preparedness programs that could (1) guide decisions on how to 
improve policies and plans that define roles and responsibilities across the 
broad spectrum of governmental and nongovernmental organizations 
involved in prevention, protection, response, and recovery activities and 
(2) help managers prioritize the use of finite resources to narrow gaps in 
needed capabilities. 

FEMA — a component in DHS — is the federal agency responsible for 
leading the nation's preparedness activities. In December 2003, the 
President issued guidance that called on the Secretary of Homeland 
Security to carry out and coordinate preparedness activities with public, 
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private, and nonprofit organizations involved in such activities. 17 In the 
wake of the problems that marked the response to Hurricane Katrina in 
2005, Congress passed the Post-Katrina Act in October 2006. The act 
strengthened FEMA's role within DHS and defined FEMA's primary 
mission as: "to reduce the loss of life and property and protect the Nation 
from all hazards, including natural disasters, acts of terrorism, and other 
man-made disasters, by leading and supporting the nation in a risk-based, 
comprehensive emergency management system of preparedness, 
protection, response, recovery, and mitigation." 18 The act required FEMA 
to establish a national preparedness system for ensuring that the nation 
has the ability to deal with all hazards, including those incidents with 
catastrophic consequences. Among other things, the act directs FEMA to 
provide funding, training, exercises, technical assistance, planning, and 
other assistance to build tribal, local, state, regional, and national 
capabilities (including communications capabilities) necessary to respond 
to any type of disaster. It also requires FEMA to develop and coordinate 
the implementation of a risk-based all-hazards strategy for preparedness 
that builds those common capabilities necessary to any type of disaster, 
while also building the unique capabilities necessary to respond to specific 
types of incidents that pose the greatest risk to the United States. 19 The 
act includes a number of other specific requirements including the 
development of quantifiable performance measurements to support each 
component of the national preparedness system, such as capabilities 
assessment, training, and exercises. 20 The system provides a basis for 
improvements in policies, plans, and capabilities that aim to save lives and 
protect and preserve property. 



Homeland Security Presidential Directive (HSPD) 8 — National Preparedness (Dec. 17, 
2003). 

18 6U.S.C. § 313(b)(1). 

19 6 U.S.C. § 313(b)(2)(G)-(H). 

20 6 U.S.C. §§ 744, 749(b). In November 2008, we reported on FEMA's and DHS's actions to 
implement the many provisions of the Post-Katrina Act. See GAO, Actions Taken to 
Implement the Post-Katrina Emergency Management Reform Act of 2006, GAO-09-59R 
(Washington, D.C.: Nov. 21, 2008). 
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DHS has defined national preparedness as a continuous cycle that involves 
four main elements: (1) policy and doctrine, (2) planning and resource 
allocation, (3) training and exercises, and (4) an assessment of capabilities 
and reporting. 21 The following is a brief description of each element of the 
system. 

Policy: This element involves ongoing management and maintenance of 
national policy and doctrine for operations and preparedness. Disaster 
response is primarily handled by local or tribal governments with the state 
and federal governments and private and nonprofit sectors playing 
supporting and ad hoc roles, respectively, as needed or as requested. 22 
One of the lessons learned from Hurricane Katrina is that a lack of clarity 
regarding roles and responsibilities across these levels of government and 
sectors can result in a less coordinated national response and delay the 
nation's ability to provide life-saving support when needed. 23 Broadly 
speaking, FEMA's role, in cooperation with other federal and nonfederal 
entities, is to define the roles and responsibilities for all response 
stakeholders so that each understands how it supports the broader 
national response. This approach calls for a national response based on 
partnerships at and across all levels. 

Planning and resource allocation: This element involves application of 
common planning processes and tools by government officials, working 
with the private sector, nongovernmental organizations, and individual 
citizens to identify requirements, allocate resources, and build and 
maintain coordinated capabilities that are prioritized based upon risk. 
Among other things, this element involves developing planning processes 
so that roles and responsibilities of stakeholders are clearly defined for 
specific homeland security scenarios, such as a hurricane or a terrorist 
attack involving nuclear or radiological weapons. 

Training and exercises: Exercises provide opportunities to test plans 
and improve proficiency in a risk-free environment. Exercises assess the 



Department of Homeland Security: National Preparedness Guidelines (Washington, 
D.C.: September 2007), p. 22. 

22 Disaster response is handled at the local level, and escalates to state or federal response 
as needed when conditions overtake local emergency responder abilities. 

23 GAO, Catastrophic Disasters: Enhanced Leadership, Capabilities, and Accountability 
Controls Will Improve the Effectiveness of the Nation's Preparedness, Response, and 
Recovery System, GAO-06-618 (Washington, D.C.: Sept. 6, 2006). 



National Preparedness 
Aims to Clarify Roles and 
Responsibilities and Close 
Gaps in Disaster Response 
Capability 
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adequacy of capabilities as well as the clarity of established roles and 
responsibilities. Short of performance in actual operations, exercise 
activities provide the best means to evaluate returns on homeland security 
investments. The Post-Katrina Act requires FEMA to carry out a National 
Training Program and National Exercise Program. 24 On January 26, 2007, 
the National Security Council and the HSC approved the establishment of 
a new iteration of the National Exercise Program to conduct exercises to 
help senior federal government officials prepare for catastrophic crises 
ranging from terrorism to natural disasters. Well-designed and executed 
exercises can improve interagency coordination and communications, 
highlight capability gaps, and identify opportunities for improvement. 25 
Tracking corrective actions resulting from exercises is a key step in the 
process. 

Assessing capabilities and reporting: According to the Post-Katrina 
Act, FEMA is required to develop a comprehensive assessment system to 
assess the nation's prevention capabilities and overall preparedness. A 
key part of the system involves the development of quantifiable standards 
and metrics — called target capabilties — that can be used to assess existing 
capability levels compared with target capability levels. 26 The act requires 
FEMA to include the results of its comprehensive assessments in an 
annual Federal Preparedness Report to Congress. To assist in this effort, 
FEMA is to receive annual State Preparedness Reports from all 50 states, 
the District of Columbia, and 5 territories that receive DHS preparedness 
assistance. 27 

FEMA's National Preparedness Directorate has primary responsibility for 
carrying out the key elements of the national preparedness system, in 
coordination with other federal, state, local, tribal, nonprofit, and private- 
sector organizations. The directorate includes the National Integration 



24 6 U.S.C. § 748. 

25 A 2006 Senate report on the federal response to Hurricane Katrina noted that a 2004 
exercise — called Hurricane Pam — illustrates the value and importance of exercises and 
tracking corrective actions in building national preparedness. The exercise resulted in 
functional plans that were considered for and actually put to use before, during, and after 
Hurricane Katrina. According to the Senate report, many of the Hurricane Pam lessons 
learned were not applied because too little was done to implement corrective actions in the 
plans resulting from the exercise. 

26 6 U.S.C. § 749. 

27 6 U.S.C. § 752(a), (c). 
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Center and the Office of Preparedness Policy, Planning, and Analysis 
(PPPA). 28 The National Preparedness Directorate and FEMA's Disaster 
Operations Directorate 29 share responsibility for ensuring plans that 
describe roles and responsibilities are developed. FEMA's National 
Exercise Division — a division of the National Integration Center — leads 
exercise activities. Finally, PPPA is responsible for assessing capabilities. 
See figure 2 for an organizational chart of select FEMA components 
involved in preparedness programs. 



FEMA created the National Preparedness Directorate as a result of the national 
preparedness system requirements established in the Post-Katrina Act. 

29 The Disaster Operations Directorate is a FEMA component separate from the National 
Preparedness Directorate. It is responsible for the coordination of all federal emergency 
management response operations; response planning; logistics programs; and integration 
of federal, state, tribal, and local disaster programs. 
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Figure 2: Selected FEMA Components Involved in Preparedness Programs 
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Source: GAO analysis. 
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FEMA faces two main challenges in developing and integrating the 
elements of the national preparedness system. 

• First, the size and complexity of the nation's preparedness activities 
and the number of organizations involved — both public and private — 
pose a significant challenge to FEMA as it leads the nation's efforts to 
develop and sustain a national preparedness system. 30 31 To develop an 
effective system, FEMA is to coordinate and partner with a broad range 
of stakeholders involved in the preparedness system. In addition, the 
Post-Katrina Act requires FEMA to coordinate with the National 
Advisory Council and the National Council on Disability when 
implementing various requirements, including establishing the National 
Exercise Program and the comprehensive assessment system. 32,33 Such 
extensive coordination requires a multidisciplinary approach involving 
planning, training, and program activities that must address challenges 
related to different organizational cultures, varying procedures and 
work patterns among organizations, and lack of communication 
between departments and agencies. The progress that FEMA makes in 
developing the preparedness system will depend, in part, on how it 



For example, see GAO, National Response Framework: FEMA Needs Policies and 
Procedures to Better Integrate Non-Federal Stakeholders in the Revision Process, 
GAO-08-768 (Washington, D.C.: June 11, 2008) and Voluntary Organizations: FEMA 
Should More Fully Assess Organization's Mass Care Capabilities and Update the Red 
Cross Role in Catastrophic Events, GAO-08-823 (Washington, D.C.: Sept. 18, 2008). 

:31 See Homeland Security Advisory Council, Top Ten Challenges Facing The Next Secretary 
of Homeland Security (Washington, D.C.: Sept. 11, 2008). Among other things, this report 
determined that the work of strengthening disaster response capabilities is incomplete, in 
part, because DHS will need to ensure involvement of homeland security partners in 
building a bottom-up approach of organization and response as it establishes national 
planning efforts. 

32 The Post-Katrina Act requires the Secretary of Homeland Security to establish a National 
Advisory Council to ensure effective and ongoing coordination of federal preparedness, 
protection, response, recovery, and mitigation for natural disasters, acts of terrorism, and 
other man-made disasters. 6 U.S.C. § 318(a). The National Advisory Council advises the 
FEMA Administrator on all aspects of emergency management, incorporating state, local, 
and tribal government and private-sector input in the development and revision of national 
preparedness policies and plans. 6 U.S.C. § 318(b). 

33 The National Council on Disability is an independent federal agency and is composed of 
15 members appointed by the President. It provides advice to the President, Congress, and 
executive branch agencies to promote policies, programs, practices, and procedures that 
guarantee equal opportunity for all individuals with disabilities, regardless of the nature or 
severity of the disability and to empower individuals with disabilities to achieve economic 
self-sufficiency, independent living, and inclusion and integration into all aspects of 
society. 



Developing and Integrating 
the Elements of the 
National Preparedness 
System Is a Challenging 
Responsibility 
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involves these preparedness stakeholders in plans that define roles and 
responsibilities, exercises, and assessments of capabilities. 

• Second, FEMA has experienced a number of organizational and staffing 
changes as a result of its incorporation into DHS, subsequent 
departmental reorganizations, and, most recently, the enactment of the 
Post-Katrina Act. Following DHS's formation, the President issued 
Homeland Security Presidential Directive 8 in December 2003, 
requiring the Secretary of Homeland Security to carry out elements of 
national preparedness, such as resource allocation to states, exercises 
and training, and capability assessments. In carrying out this directive, 
non-FEMA components in DHS conducted preparedness activities. In 
July 2005, the Secretary of Homeland Security separated preparedness 
programs from response and recovery. All preparedness programs, 
including planning, training, exercising, and funding, were placed in a 
new DHS Directorate for Preparedness. Meanwhile, FEMA was a 
separate component that was responsible for response and recovery. 
In October 2006, the Post-Katrina Act reversed this organizational 
change, restoring to FEMA most of the functions that the Preparedness 
Directorate had assumed, effective March 31, 2007. 34 As a result of this 
statutory transfer of authority, FEMA inherited the preparedness 
programs that had been carried out by other elements of DHS pursuant 
to HSPD-8. For example, the National Exercise Program was moved to 
FEMA from DHS's Office of Grants and Training in January 2007. 36 
Similarly, parts of the comprehensive assessment system were 
previously carried out by non-FEMA components in DHS, according to 
FEMA officials. For example, target capabilities were developed by the 
Office of Policy Analysis and other capability assessments were 
developed by DHS programs outside of FEMA. 38 In January 2008, 
FEMA established the PPPA component within the National 
Preparedness Directorate to develop the comprehensive assessment 
system. Such organizational changes have resulted in turnover of staff 
and loss of institutional knowledge of previous efforts. For example, in 



M 6 U.S.C. § 315. 

35 The Exercise Program has undergone five organizational changes since 2003. The 
program was in the Department of Justice before it was transferred to DHS in 2003, at 
which time it was initially placed in another DHS component called the Office of Domestic 
Preparedness. 

'"'DHS's Office of Policy and Analysis developed a Web-based assessment that was piloted 
in 10 states (known as the National Preparedness System). DHS's Office of State and Local 
Government Coordination and Preparedness developed an in-person assessment that was 
piloted in 6 states (known as a Pilot Capability Assessment). 
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March 2008 half of the staff positions in the PPPA office were vacant 
and these positions were not filled until November 2008, according to 
FEMA officials. In addition, FEMA did not hire a permanent director 
for the PPPA office until October 2008. Reorganization of the National 
Exercise Program also brought about staffing changes, according to 
FEMA officials. Among other things, in the fall of 2008 FEMA hired a 
new director for the National Exercise Division — a position that had 
been vacant for several months, according to FEMA officials. 37 Fig. 3 
illustrates the organizational changes related to the National Exercise 
Program. 



In addition to hiring a new director, FEMA had filled 80 percent (20 of 25) of the budgeted 
positions for the program by October 2008, according to FEMA officials. 
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Figure 3: Organizational Changes to the National Exercise Program (2001-2008) 



9/11 



DHS 
Established 



Hurricane 
Katrina 



Post- 
Katrina 
Act 



2001 



2002 



2003 



2004 



2005 



2006 



Department of Justice 



The Exercise Program is in 
the Office of Domestic 
Preparedness. 



2007 



2008 



I 



Department of 
Homeland Security (DHS) 

The Office of Domestic 
Preparedness and the Exercise 
Program are transferred to DHS. 

The Office of Domestic 
Preparedness becomes the 
Office of State and Local 
Government Coordination and 
Preparedness. 



The Office of State and Local 
Government Coordination and 
Preparedness becomes the Office 
of Grants and Training. 



Federal Emergency 
Management Agency 
(FEMA) 



FEMA is moved into DHS 



The National Preparedness 
Directorate (NPD) is created 
in FEMA and the Office of 
Grants and Training is 
transferred to FEMA. 

Grants are separated from 
training and exercises and 
moved to a new Grant 
Programs Directorate. 

Training and exercises are 
moved to the National 
Integration Center, a 
subcomponent in NPD. 



Source: Department of Homeland Security and GAO analysis. 



Page 22 



GAO-09-369 National Preparedness 



Most Policies That 
Define Roles and 
Responsibilities and 
Planning Processes 
Have Been 
Completed, but Work 
Remains to Complete 
Emergency Plans 



Defining roles and responsibilities is a key step in developing the national 
preparedness system. While most key policies that define roles and 
responsibilities have been completed, 68 percent (49 of 72) of plans that 
operationalize such policies have not been completed. Lessons learned 
from Hurricane Katrina and emergency response exercises demonstrated 
the need for the development of complete policies and plans to address 
potential problems with stakeholders not understanding their roles and 
responsibilities in response to a catastrophic disaster. Although best 
practices for program management state that a program management plan 
is an essential tool for implementing a program, FEMA, in coordination 
with DHS and other federal entities, has not yet fully developed such a 
plan to help ensure the development and integration of policies and plans 
that define roles and responsibilities and planning processes for 
emergency response. 



Defining Roles and 
Responsibilities Is Key to 
Developing the 
Preparedness System 



Legislation and presidential directives call for the development of policies 
and plans that define roles and responsibilities, which is key to FEMA's 
ability to develop the preparedness system. For example, the Homeland 
Security Act of 2002, as amended by the Post-Katrina Act, requires FEMA 
to consolidate existing federal government emergency response plans into 
a single, coordinated National Response Plan, now known as the National 
Response Framework. 38 To develop the preparedness system, FEMA is to 
partner and coordinate with key stakeholders, including other DHS 
components as well as other federal, state, and local entities. For 
example, the Post-Katrina Act requires the FEMA Administrator, under the 
leadership of the Secretary of Homeland Security, to coordinate with other 
agencies and offices in DHS to take full advantage of their range of 
available resources. 39 In addition, the Post-Katrina Act requires FEMA to 
coordinate with other federal departments and agencies and the National 
Advisory Council to develop and refine key national policy documents that 



J8 6 U.S.C. § 314(a)(6). DHS issued the National Response Plan (NRP) in December 2004 
and made some revisions to the plan in May 2006 to address lessons learned from 
Hurricane Katrina, such as a lack of clarity in federal leadership roles and responsibilities 
which resulted in disjointed and delayed efforts by emergency responders. The January 
2008 National Response Framework (NRF) replaced the NRP. For the purposes of this 
report, the base NRF document, Emergency Support Function (ESF) Annexes, Support 
Annexes, and Partner Guides are identified as policies that define roles and 
responsibilities. The NRF also contains seven Incident Annexes and two Incident Annex 
Supplements, which for the purposes of this report are identified as plans because these 
incident annexes and supplements are scenario-specific plans. 

39 6 U.S.C. § 313(b)(2)(F). 
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broadly define roles and responsibilities, including the National Incident 
Management System (NIMS) and the NRF. 40 

Plans developed using established planning processes operationalize 
policy documents, such as the NRF, by providing additional details on the 
roles and responsibilities for each individual and organization that may be 
involved in responding to high-risk or catastrophic incidents. 41 For 
example, during the planning process, FEMA may need to coordinate with 
other federal departments and agencies — such as the Department of 
Defense (DOD), Federal Bureau of Investigation, and other components in 
DHS — to define roles and responsibilities for responding to a chemical 
attack. Per the Post-Katrina Act, each federal agency with responsibilities 
under the NRF is responsible for developing its own operational plans. 42 

The act further requires the President to certify to selected committees of 
Congress on an annual basis that each federal agency has complied with 
statutory requirements in the development of its operational plan, 
including a requirement that the plan "be coordinated under a unified 
system with a common terminology, approach, and framework." 43 
Although the Post-Katrina Act does not charge FEMA with developing or 
certifying the federal agency plans, FEMA is statutorily responsible for the 
basic architecture of the national preparedness system, in coordination 
with other federal departments and agencies, among others. This 
principle of integration and coordination is also embodied in Homeland 
Security Presidential Directive 8 Annex 1 (HSPD 8 Annex 1), which tasked 
DHS with developing a national planning system to develop and integrate 
plans that define preparedness roles and responsibilities, both horizontally 
across federal departments and agencies (i.e., integration of plans that 
have been developed by more than one federal department or agency) and 
vertically with state and local emergency response plans (i.e., integration 



40 6 U.S.C. § 319(b)(1); see also 6 U.S.C. § 745. 

41 High-risk incidents include those that are defined in the National Planning Scenarios, 
which represent examples of the gravest dangers facing the United States, including 
terrorist attacks and natural disasters, and have been accorded the highest priority for 
federal planning efforts. A catastrophic incident is any natural or man-made incident, 
including terrorism, which results in extraordinary levels of mass casualties, damage, or 
disruption severely affecting the population, infrastructure, environment, economy, 
national morale, and/or government functions. 

42 6 U.S.C. § 753(a)(4). 

43 6 U.S.C. § 753(b)(1), (d). 
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of plans that have been developed by more than one level of government). 
Thus, the federal preparedness framework depends on DHS's — and, in 
particular, FEMA's — ability to coordinate with its federal department and 
agency partners to ensure that their plans are integrated in a way that 
avoids duplication of effort and confusion during an interagency response. 



DHS, FEMA, and other federal entities with a role in national preparedness 
have completed most of the key policies that broadly define roles and 
responsibilities and planning processes for developing more detailed 
emergency plans. 44 Among the 50 policies that define roles and 
responsibilities or planning processes, 42 have been completed, 2 have 
been partially completed, and the remaining 6 are incomplete. 

A more detailed breakdown of the 50 policies shows that 46 define roles 
and responsibilities and 4 define planning processes for developing 
emergency plans. Forty of the 46 policies that define roles and 
responsibilities have been completed. 45 Among the policies that have been 
completed, for example, DHS issued the revised NIMS in December 2008 
to further clarify roles and responsibilities when multiagency, 
intergovernmental entities are involved in a response and to address, in 
part, the confusion about roles and responsibilities that resulted in a poor 
response to Hurricane Katrina. 46 Key components of the NRF have also 
been completed, including the base NRF document, 15 Emergency 
Support Function Annexes, and 8 Support Annexes. 47 However, other 
components of the NRF have not been completed, such as the 4 Partner 
Guides that are to provide abbreviated descriptions of the key roles and 
responsibilities of specific federal, state, local, and private sector and 



Most Key Policies That 
Define Roles and 
Responsibilities and 
Planning Processes Have 
Been Completed 



For details on all the policies and plans identified in this section of the report, see app. II. 

45 We considered policies to be completed if they have been published in final form. 

46 NIMS presents, among other things, doctrine that standardizes the process for emergency 
response stakeholders to conduct integrated emergency management and incident 
response operations by establishing organizational incident management structures. NIMS 
structures enable incident action planning during an event response. 

47 The NRF Emergency Support Function Annexes align categories of federal government 
response resources and capabilities and provide strategic objectives for their use under the 
NRF. The NRF Support Annexes describe the roles and responsibilities of federal 
departments and agencies and nonfederal entities in coordinating and executing the 
common functional processes and administrative requirements necessary for incident 
management that are common to all incidents. 
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nongovernmental stakeholders under the NRF. 48 Also uncompleted is the 
development of new Joint Field Office (JFO) guidance under the NRF 
which, according to FEMA officials, is to provide functional guidance for 
the organization and staffing of JFOs, as well as their establishment, 
operation, and demobilization. 49 In addition, DHS has not completed the 
National Homeland Security Plan, which is to serve as an overarching 
strategic plan to guide national efforts to execute the National Strategy for 
Homeland Security. For details on, and the status of, all 46 policies that 
are to define roles and responsibilities, see table 4 in appendix II. 

In addition to the 46 policies that define roles and responsibilities, 4 other 
policies are to define planning processes for developing emergency 
plans. 50 Two of these 4 policies have been completed. According to FEMA 
officials, FEMA issued planning guidance for its operational planners in 
March 2009 and will update this guidance each fiscal year. The other 2 
policies that define planning processes have been partially completed — 1 
has been issued as an interim policy (Comprehensive Preparedness Guide 
301) while the other has been drafted (Integrated Planning System) and is 
being used by federal interagency incident management planners although 
it has not been publicly released. 51 Additional details describing these 4 
policies and their status are included in table 5 in appendix II. 



48 The four NRF Partner Guides are to summarize core NRF concepts and be tailored 
specifically to leaders at different levels of government and from different types of 
organizations. The NRF Partner Guides comprise four of the six policies that define roles 
and responsibilities that have not been completed. According to FEMA officials, the four 
Partner Guides have been drafted and are awaiting revision pending guidance from FEMA 
leadership. 

49 The JFO is a temporary federal multiagency coordination center established locally to 
facilitate field-level domestic incident management activities. Three existing JFO policy 
guides, developed prior to the issuance of the NRF and not yet revised, are designed to 
assist personnel assigned to response operations. 

°°Our September 2006 report on Hurricane Katrina recommended that DHS provide 
guidance and direction for federal, state, and local planning efforts to ensure such activities 
fully support preparedness, as well as take a lead in monitoring federal agencies' efforts to 
meet their responsibilities. See GAO-06-618. 

ol The Comprehensive Preparedness Guide 101 provides state and local communities with 
guidance for emergency operations planning and describes how the state and local 
planning process is to vertically integrate with the federal Integrated Planning System. The 
Comprehensive Preparedness Guide 301 is intended to be a tool for state, territorial, tribal, 
and local emergency managers in the development of emergency operations plans that 
address planning for special needs populations. The Integrated Planning System, among 
other things, is to provide common processes for developing emergency plans. 
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Although Action Has Been 
Taken to Develop and 
Complete Some Plans That 
Define Roles and 
Responsibilities, 68 
Percent of the Plans Have 
Not Been Completed 



While DHS, FEMA, and other federal entities with a role in national 
preparedness have taken action to develop and complete some plans that 
detail and operationalize roles and responsibilities for federal and 
nonfederal entities, these entities have not completed 68 percent of the 
plans required by existing legislation, presidential directives, and policy 
documents as of April 2009. 52 Specifically, of the 72 plans we identified, 20 
have been completed (28 percent), 3 have been partially completed (that 
is, an interim or draft plan has been produced — 4 percent), and 49 (68 
percent) have not been completed. Detailed plans supplement and 
operationalize key policy documents. Among the plans that have been 
completed, FEMA published the Pre-Scripted Mission Assignment Catalog 
in 2008, which defines roles and responsibilities for 236 mission 
assignment activities to be performed by federal government entities, at 
the direction of FEMA, to aid state and local jurisdictions during a 
response to a major disaster or an emergency. 53 See table 6 in appendix II 
for additional details on the other 19 plans that have been completed. 



One of the three plans that FEMA has partially completed is the Federal 
Contingency Plan — New Madrid Seismic Zone Catastrophic Earthquake. 54 
This plan addresses major issues the federal government expects to 
encounter if a catastrophic earthquake occurs in the New Madrid Seismic 



We determined that 16 of the 72 plans that need to be developed are operational plans 
required by HSPD 8 Annex 1. We calculated this number by counting DHS and FEMA as 
entities that both need to develop operational plans to carry out their roles and 
responsibilities for each of the eight consolidated national planning scenarios under the 
planning process established by Annex 1. In addition to DHS and FEMA, it is likely that 
other federal departments and agencies will also have to develop operational plans to 
address their roles and responsibilities in response to each of the eight consolidated 
national planning scenarios, as appropriate. For example, other federal entities, such as 
DOD, EPA, and the Coast Guard responded to Hurricane Katrina and would likely need to 
develop an operational plan per Annex 1 for response to a major hurricane. As of April 
2009, the exact number of operational plans that federal departments and agencies will 
need to develop as required by Annex 1 is unknown. It is, however, likely to be higher than 
the 16 we used in our calculation. 

53 A mission assignment is a work order issued by FEMA to another federal agency which 
directs completion by that agency of a specified task and sets forth funding, other 
managerial controls and guidance. 6 U.S.C. § 741(4). To expedite the provision of federal 
assistance, the Post-Katrina Act required FEMA to develop prescripted mission 
assignments in coordination with other relevant federal agencies. 6 U.S.C. § 753(c). 

54 For the purposes of this report, we identified plans as "partially completed" if they have 
been issued in interim or draft formats, but not finalized. In addition to the interim Federal 
Contingency Plan — New Madrid Seismic Zone (NMSZ) Catastrophic Earthquake, we 
identified two other plans as partially completed: the draft Hawaii Hurricane Contingency 
Plan and the draft Northwest Nevada Earthquake Contingency Plan. 
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Zone with no warning. FEMA published this plan in an interim form in 
June 2008 and intends to finalize it by May 2010. While FEMA has engaged 
in significant planning efforts regarding threats that are specific to certain 
regions, such as hurricanes and earthquakes, through its Catastrophic 
Disaster Planning Initiative, those planning efforts are ongoing and have 
not been concluded. 55 See table 6 in appendix II for additional details on 
the other two plans that have been partially completed. 

Among the 49 plans that have not been completed are the NRF incident 
annexes for terrorism and cyber incidents as well as the NRF incident 
annex supplements for catastrophic disasters and mass evacuations. The 
NRF incident annexes and incident annex supplements are to address the 
roles and responsibilities and unique aspects of how the United States 
responds to broad incident types. 56 In addition, operational plans for 
responding to the consolidated national planning scenarios, as called for in 
HSPD 8 Annex 1, remain outstanding. For additional details and the status 
of each of these plans, as well as other plans that are to define roles and 
responsibilities, see table 6 in appendix II. 

Developing plans to operationalize policies that define roles and 
responsibilities is one key to an effective response. According to DHS, 
effective response hinges upon well-trained leaders and responders who 
have invested in response preparedness, developed engaged partnerships, 
and are able to achieve shared objectives. Until outstanding policies and 
plans, especially those that are new, are completed, FEMA, in 
coordination with DHS and other federal departments and agencies, 
cannot provide associated training on such policies and plans, and, 
relatedly, cannot validate the new policies, plans, and training though 
exercises — the next step in the national preparedness system cycle. 



Tor example, the New Madrid Seismic Zone planning efforts under the FEMA 
Catastrophic Disaster Planning Initiative have included, among others: earthquake 
response capability assessments for each of the eight NMSZ states; facilitated earthquake 
response operations planning sessions in 30 FEMA-supported workshops, conducted with 
the 747 counties and the eight states in the NMSZ; and participation by more than 3,800 
representatives of federal, state, tribal, local, and county emergency management and 
responder organizations, as well as the private sector. 

56 NRF incident annexes were released, from June through November 2008, for five of seven 
incident scenarios (Biological; Catastrophic; Food and Agriculture; Mass Evacuation; and 
Nuclear/Radiological). The Terrorism Incident Law Enforcement & Investigation Annex, 
Cyber Incident Annex, Catastrophic Incident Supplement (NRF-CIS) to the Catastrophic 
Incident Annex (NRF-CIA), and Mass Evacuation Incident Annex Operational Supplement 
have not been released as of April 2009. 
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According to FEMA, certain plans that have yet to be completed are 
refinements of existing plans, and training has been provided on the 
existing plan. However, such existing training will need to be adapted and 
modified to enable response stakeholders to be trained on the revised 
plans. Incomplete policies and plans, especially those that are new, and 
the resulting lack of associated training and validation through exercises, 
increase the risk that response to an incident may be disjointed, delayed, 
or ineffective because stakeholders may not understand their roles and 
responsibilities. 

The issue of completing emergency policies and plans that define roles 
and responsibilities is not new. Prior to Hurricane Katrina, based on a 
summary of lessons learned from exercises conducted in fiscal year 2005, 
DHS determined that plans for specific incidents were potentially needed 
to clarify how the National Response Plan (now the NRF) would be 
implemented under several types of domestic scenarios, such as the threat 
of an improvised nuclear device detonation, large-scale biological events, 
and suicide bombings. In February 2006, the White House issued its report 
on lessons learned from the response to Hurricane Katrina and concluded 
that two of four critical deficiencies in the response involved policies and 
plans, or the lack thereof, that were to detail roles and responsibilities. 57 
Among other things, the report noted that federal departments and 
agencies were required to develop supporting operational plans and 
standard operating procedures for national response activities, but in 
almost all cases, these required plans and procedures were either 
nonexistent or still under development. In addition, the report stated that 
additional structural deficiencies in the national preparedness system 
included weak regional response planning and coordination structures. 58 
Further, in our September 2006 report on Hurricane Katrina, we 
recommended that FEMA develop detailed and robust operational 
implementation plans for the National Response Plan (now NRF) and its 
Catastrophic Incident Annex and Supplement in preparation for and 
response to future catastrophic disasters. 59 



Two additional deficiencies cited by the White House report were the failure of the 
federal government to properly execute the processes designed to provide unified 
management of the national response for all federal and nonfederal stakeholders, as well as 
the failure of command and control structures for federal government operations. 

58 The White House, The Federal Response to Hurricane Katrina: Lessons Learned 
(Washington, D.C.: Feb. 23, 2006). 

59 GAO-06-618. 
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In addition, in October 2006, the Post-Katrina Act required FEMA to 
develop prescripted mission assignments and each federal agency with 
NRF responsibilities to develop operational plans and corresponding 
capabilities in support of the NRF to ensure a coordinated federal 
response. 60 In December 2007, the President signed HSPD 8 Annex 1, 
which called for the development of an integrated set of scenario-based 
response plans, including federal agency operational plans that are also 
needed to satisfy the Post-Katrina Act. More recently, in September 2008, 
the Homeland Security Advisory Council (HSAC) 61 for DHS identified the 
work of strengthening the nation's disaster response capabilities, which 
includes developing and publishing outstanding emergency response 
policies and plans that define roles and responsibilities for national 
preparedness, as 1 of the top 10 DHS challenges for the incoming 
presidential administration. 62 



60 6 U.S.C. § 753(a)(4), (c). 

61 The HSAC operates in accordance with the provisions of the Federal Advisory Committee 
Act (FACA), Title 5 United States Code, Appendix, to provide advice and recommendations 
to the Secretary of Homeland Security on matters pertaining to DHS strategy, policy, 
leadership, coordination, management, implementation, evaluation, and feedback. 

62 Among other issues, the HSAC has identified the need for the new Secretary of Homeland 
Security to focus additional attention on certain response issues, including the national 
planning system, implementing the NRF and NIMS, as well as developing a framework for 
catastrophic disaster recovery in addition to ongoing preparedness and response planning 
efforts. 
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Despite Exercises 
Demonstrating the Need 
for the Development of 
Complete Policies and 
Plans, FEMA Has Not Fully 
Developed a Program 
Management Plan for Their 
Development and 
Integration 

Recent Exercises Have I R addition to the lessons learned from Hurricane Katrina, recent exercises 

Demonstrated the Importance nave demonstrated the ongoing need for the development of complete 
of the Development of policies and plans that define roles and responsibilities for national 

Complete Policies and Plans to preparedness. For example, we reviewed 16 after-action reports for 
Define Roles and national or principal level or equivalent exercises that were conducted 

Responsibilities from 2005 to 2008, and all of the reports (16 of 16) called for further 

clarification of roles and responsibilities across federal departments and 
agencies or between federal and nonfederal organizations, as shown in the 
following examples. 

• As a result of a February 2007 exercise that tested a joint response by 
federal agencies to an attack using improvised explosive devices, HSC 
determined that the federal government needed to further refine the 
delineation of roles and responsibilities for the Departments of 
Defense, Justice, and Homeland Security. The same exercise also 
identified the need for the federal government to increase outreach to 
governors in order to achieve a common understanding of federal and 
state governments' respective roles and responsibilities during such an 
incident. Finally, this exercise demonstrated that policies and plans 
were needed to clearly articulate, prior to an incident occurring, the 
circumstances in which the use of military assets and resources for 
civilian response is appropriate. Not addressing these recommended 
corrective actions, among others, may result in less efficient and 
effective responses. 

• In a September 2007 exercise, the HSC identified the need for senior 
federal officials to be rapidly informed of the possible courses of action 
available to them in responding to a major disaster or an emergency. 
To do so, HSC asserted that the senior federal officials' respective 
organizations should have in place detailed plans to inform and execute 
the senior federal officials' decisions, as such plans would serve to 
better integrate interagency response activities. HSC made this 
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recommendation in September 2007, and it was acted on by the 
President through the establishment of HSPD 8 Annex 1 in December 
2007, more than a year after the Post-Katrina Act called for the 
development of federal agency operational plans. 63 

FEMA Has Not Established a 
Program Management Plan for 
the Development and 
Integration of Outstanding 
Emergency Response Policies 
and Plans 



Best practices for program management, established by the Project 
Management Institute in The Standard for Program Management, state 
that managing a program includes, among other things, (1) establishing 
clear and achievable objectives; (2) balancing the competing demands for 
quality, scope, time, and cost; and, (3) adapting the specifications, plans, 
and approach to the different concerns and expectations of the various 
stakeholders involved in the program's projects. 65 A key step in managing 
a program involves developing a program management plan, which is an 
approved document that defines how a program will be executed, 
monitored, and controlled. The program management plan defines the 
tactical means by which the program will be carried out. According to The 
Standard for Program Management, a program management plan should, 
among other things: 



FEMA has not established a program management plan, in coordination 
with DHS and other federal entities, to ensure the development and 
integration of outstanding policies and plans that are to define roles and 
responsibilities and planning processes. The Post-Katrina Act makes the 
FEMA Administrator, in coordination with other entities, responsible for 
the development of the national preparedness system. According to the 
National Preparedness Guidelines, the national preparedness system, in 
part, consists of the policies and plans that define roles and 
responsibilities and planning processes for developing emergency plans. 
Although the Post-Katrina Act requires federal agencies to develop their 
own operational plans, those plans are to be "coordinated under a unified 
system with a common terminology, approach, and framework." 64 This 
coordination and unification is central to FEMA's mission as the lead 
agency in charge of national preparedness, and it requires that the policies 
and plans that have been called for are developed and integrated so that 
emergency response roles and responsibilities and planning processes are 
fully defined and implemented. 



The Post-Katrina Act was signed into law by the President in October 2006. 
64 6 U.S.C. § 753(b)(1). 

65 Project Management Institute. The Standard for Program Management, 2 nd ed. (Newton 
Square, Pa.: 2006.) 
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• identify the specific schedule of activities that need to be performed to 
complete and identify dependencies among policy and planning 
development activities; 

• identify the types and quantities of resources required to perform, and 
amount of time needed to complete, all policy and planning 
development activities; 

• analyze activity sequences, durations, resource requirements, and 
schedule constraints to create and update the policy and planning 
project schedules; and 

• control for changes to the project schedules precipitated by outside 
forces. 

Because FEMA has not established, in coordination with DHS and other 
federal entities, such a plan to ensure the development and integration of 
outstanding policies and plans, it is unclear when the full set of policies 
and plans will be completed, and FEMA cannot determine whether it or 
other entities with policy and plan development responsibilities, such as 
DHS, are on schedule. Further, because FEMA cannot determine whether 
other entities with policy and plan development responsibilities are on 
schedule, it cannot determine when and how it will integrate into the 
national preparedness cycle the range of policies and plans required by 
legislation, presidential directives, and other policy. Based in part on our 
work, in February 2009 FEMA officials acknowledged that a program 
management plan should be established. Without active utilization of a 
program management plan, FEMA, in coordination with DHS and other 
federal entities, may experience unforeseen delays in completing its 
efforts to develop and integrate these policies and plans that define roles 
and responsibilities and planning processes. While FEMA, in coordination 
with DHS and other federal entities, may experience unanticipated or 
uncontrollable delays in developing outstanding policies and plans, it 
would be better positioned to identify the effect of those delays and assess 
measures to mitigate them with a program management plan in place and 
utilized. 
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FEMA Has Taken 
Actions to Implement 
the National Exercise 
Program, but Faces 
Challenges in Meeting 
Statutory and 
Program 
Requirements 



FEMA has developed guidance to implement the National Exercise 
Program; however, it faces challenges in meeting statutory and program 
requirements in conducting the program. These challenges have arisen 
because FEMA lacks procedures that detail how it will work with federal 
entities and monitor states to ensure these entities carry out program 
requirements. In addition, FEMA faces challenges in measuring the 
effectiveness of the program because the databases it uses to measure 
program performance are incomplete. 



FEMA Has Developed 
Guidance and Other Tools 
to Assist in Designing and 
Implementing the National 
Exercise Program 



Exercises are a key element of the national preparedness system. The 
purpose of the National Exercise Program is to test and improve the 
nation's ability to prevent, prepare for, and respond to events such as 
terrorist attacks and natural and man-made disasters. To meet this 
purpose, exercises should test existing capabilities against desired, or 
target capabilities as well as verify and validate policies and plans that 
define roles and responsibilities. 



Developing and implementing the National Exercise Program is a difficult 
task because the magnitude of the effort involves coordinating with and 
relying on the cooperation of other DHS components such as the Coast 
Guard, numerous federal entities such as the Homeland Security Council 
(HSC) — which is responsible for coordinating federal interagency 
homeland security policy — and state governments, among others. This 
coordination is especially critical at the federal level because FEMA lacks 
the authority to compel federal agencies to comply with program 
requirements. At the state level, FEMA is able to use its grant programs to 
ensure that states follow program guidelines. Since 2007, FEMA has taken 
a number of actions to implement the National Exercise Program. Several 
of these actions are summarized in table 1. 
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Table 1 : FEMA's Actions to Implement the National Exercise Program 



Actions taken Summary 

Executed an implementation plan" for nine The implementation plan establishes the foundation for the National Exercise Program by 
federal departments or agencies" outlining policy and guidance for federal interagency coordination and participation and 

establishes four tiers of exercises. (See fig. 6 for more details on Tier I — IV exercises.) 
The plan requires that FEMA develop and annually revise a 5-year schedule for Tier I 
and Tier II exercises, that senior officers of the federal government participate in exercise 
activities, that federal departments and agencies budget and plan for exercise 
participation, and that federal departments, agencies, or offices responsible for 
coordinating exercises adhere to the principles of the National Exercise Program. 

Refined Homeland Security Exercise and This is the primary guidance for all four tiers of exercises. FEMA requires that entities 
Evaluation Program (HSEEP) guidance that receive Homeland Security Grant Program (HSGP) funding for their exercises to 

adhere to specific HSEEP guidance for exercise program management, design, conduct, 
evaluation, and improvement planning (FEMA revised HSEEP in February 2007). 

Developed and refined tools • The National Exercise Schedule (NEXS) system is the nation's online tool that 

facilitates scheduling and synchronization of national, federal, state, and local level 
exercises. (FEMA relaunched NEXS in November 2006; a previous Exercise 
Scheduling and Reporting System was originally released in June 2004.) 

Secure Portal is a Web-based repository for federal and state exercise program 
managers' plans, after-action reports, and improvement plans. (The FEMA Secure 
Portal was fully implemented by June 2004; it was originally procured in 2002.) 

The Corrective Action Program (CAP) system is a Web-based database for tracking 
capability-based improvement plans. (FEMA launched it in March 2007.) 

The HSEEP toolkit is a collection of tools to assist managers in exercise scheduling, 
design, and evaluation. These provide users with templates and guidance for 
conducting exercises. (FEMA updated it in February 2008.) 

The Lessons Learned Information Sharing (LLIS) system is an online library of 
homeland security-related documents, including lessons learned and after-action 
reports, which are voluntarily submitted by federal and state officials. (FEMA 
launched LLIS in April 2004.) 



Source: GAO, based on FEMA information. 

"On January 26, 2007, the National Security Council and HSC unanimously reached agreement on a 
charter for the National Exercise Program and on April 1 1 , 2007, the President approved the National 
Exercise Program's Implementation Plan. As of February 2009, the plan had been revised, but had 
not yet been approved by the White House, according to FEMA officials. Our report will refer to the 
2007 plan unless otherwise noted. 

"Seven departments ( the Departments of Homeland Security, Defense, Energy, Justice, 
Transportation, State, and Health and Human Services) and two agencies (the Federal Bureau of 
Investigation and the Office of the Director of National Intelligence) are signatories to the National 
Exercise Program's Implementation Plan. 

FEMA has also identified four tiers of exercises that comprise the National 
Exercise Program (see fig. 4). Among the exercises that involve federal 
interagency coordination are Tier I National Level Exercises, which are 
operations-based exercises that evaluate existing national plans and 
policies, in concert with federal and nonfederal entities; and Principal 
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Level Exercises, which are discussion-based exercises among senior 
federal officials that examine emerging issues. 



Figure 4: Description of Tiers l-IV Exercises 




White House directed, 
U.S. government-wide strategy and 
policy focus, full federal agency 
participation. Five Tier I exercises conducted 
annually (One National Level Exercise and 
four Principal Level Excercises). 

Federal department/agency directed, 
U.S. government-wide strategy and policy 
focus. Full federal agency participation in 
person or through simulation. No more than 
three Tier II exercises conducted annually. 

Other federal exercises, regional 
operational, tactical or organizational-specific 
focus. Full federal agency participation is 
optional. The number of exercises conducted 
annually depends upon individual entities' 
needs and requirements. 

Non-federal exercises, state, territorial, local, 
and/or tribal governments and/or private 
sector focus. Federal agency participation is 
optional. The number of exercises conducted 
annually depends upon individual entities' 
needs and requirements. 



Source: FEMA. 



Officials at all six states we visited cited actions taken by FEMA to 
implement the National Exercise Program as positive contributions to 
their exercise efforts. For example, officials in five of the six states 
indicated that HSEEP guidance was beneficial because it establishes 
consistency in how exercises are designed and conducted. State officials 
who were involved in a National Level Exercise indicated that it not only 
allowed them to test and validate local emergency response plans, but also 
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gave them the opportunity to meet federal stakeholders as well as first 
responders from neighboring counties and cities, thus enabling them to 
establish working relationships before an event occurs. Similarly, exercise 
planners in New York noted that a benefit of FEMA's regional training and 
exercise plan workshop was the positive relations that were developed 
withFEMA officials. 66 



States Have Also Taken 
Actions to Implement 
Exercise Programs 



In addition to actions by FEMA, the six states we visited have also taken 
actions to implement exercise programs (FEMA considers exercises 
conducted by states to be Tier IV exercises.) For example, exercise 
planners in Washington conducted a half-day senior leadership workshop 
for federal, state, and local officials in May 2008 that identified a number 
of issues related, among other things, to intergovernmental 
communications during an incident. Based on the workshop, officials in 
Washington identified corrective actions that could be taken. For example, 
they identified the need to increase postdisaster communications to help 
speed recovery efforts and said they were taking corrective actions to 
address this issue. In California, state exercise planners involved planners 
from other departments or other states in their design and planning 
workshops as a way to share information on exercise design and conduct, 
thereby increasing the interagency participation in their exercise program 
efforts. According to the workshop sponsors, participants benefited from 
the multidisciplinary participation in the conference. In Illinois, the state 
used real-world events, such as sporting events, to test or "exercise" 
response by local law enforcement and first responders. State exercise 
officials said these efforts were an effective supplement to their regular 
exercise program. 



66 In 2008, FEMA staff conducted training and exercises plan workshops for federal and 
state government officials involved in exercises in all 10 FEMA regions. Among other 
things, stakeholders participated in break-out sessions to discuss exercises that they 
planned to conduct in 2009 and 2010. Staff used this information to help participants plan 
their exercises in coordination with other entities and to update participants on current 
events for the National Exercise Program. 
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FEMA Faces Challenges in 
Meeting Statutory and 
Program Requirements in 
Conducting and Measuring 
the Effectiveness of the 
National Exercise Program 



FEMA Does Not Have 
Procedures to Help Ensure that 
After-Action Reports from 
National and Principal Level 
Exercises Are Promptly 
Prepared and That Corrective 
Actions Are Tracked and 
Implemented 



While FEMA has taken actions to implement the National Exercise 
Program, it faces challenges in meeting statutory and program 
requirements in conducting and measuring the effectiveness of the 
program. First, FEMA does not have procedures in place to detail how it 
is to (1) work with other federal entities and (2) monitor states to help 
ensure that these entities promptly prepare after-action reports and track 
and implement corrective actions for federal- and state-level exercises. 
Second, the National Exercise Program's ability to simulate a catastrophic 
event to strain, or "stress", the preparedness system is limited. Third, 
FEMA lacks data to measure the effectiveness and progress of the 
National Exercise Program. 

FEMA must design the National Exercise Program to provide, among 
other things, for the prompt development of after-action reports and plans 
for quickly incorporating lessons learned into future operations. 67 In 
addition, the Post-Katrina Act requires FEMA to establish a program to 
conduct remedial action tracking and long-term trend analyses. 68 
According to the implementation plan for the National Exercise Program, 
after-action reports for Tier I exercises (National Level Exercises or 
Principal Level Exercises) must be issued within 180 days, or 6 months, 
after the completion of an exercise, and the release of an after-action 
report should not be delayed to reach consensus on all issues identified 
during the exercise. FEMA executes Tier I exercises in coordination with 
HSC and others. Although the Post-Katrina Act does not give FEMA the 
authority to compel other federal entities to comply with the objectives of 
the National Exercise Program, it places responsibility for implementing 
the National Exercise Program on FEMA, in coordination with other 
appropriate federal agencies and other entities. Therefore, it is incumbent 
on FEMA to coordinate with HSC and other federal entities to better 
ensure that FEMA obtains the information it needs to meet its statutory 
responsibility to track corrective actions. However, FEMA has not 
ensured that after-action reports for Tier 1 exercises are issued in a 
prompt manner, nor has it tracked and documented implementation of 
corrective actions for such exercises. These challenges occurred, in part, 
because FEMA has not established procedures that detail how the agency 



67 6 U.S.C. § 748(b)(2)(A)(vi). 

68 6 U.S.C. § 750. Our report will generally refer to "corrective action tracking" rather than 
"remedial action tracking," as used in the statute because the database FEMA uses to 
implement the tracking requirement is called the CAP system. 
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will work with other federal entities to ensure National Exercise Program 
requirements are met. 

For example, FEMA conducted a Tier I National Level Exercise — Top 
Officials 4 (TOPOFF 4)— in October 2007, but as of February 2009, or more 
than 15 months later, FEMA had not yet issued the after-action report or 
tracked and implemented corrective actions. 69 When an after-action report 
on an exercise is delayed and not provided to stakeholders, the "lessons 
learned" from the exercise diminish in importance, limiting stakeholders' 
ability to make improvements in preparedness. 70 In February 2009, FEMA 
officials stated that the draft after-action report for TOPOFF 4 had been 
written, and various departments and agencies have approved the report, 
but it was not approved by DHS prior to the change in administration 
related to the 56 th Presidential Inauguration. 71 According to FEMA 
officials, a complicating factor in releasing the report is the political 
sensitivity of information, as those who write after-action reports may face 
internal pressure not to identify weaknesses in an entity's emergency 
preparedness. As a result of these delays, stakeholders may not be able to 
promptly make improvements in preparedness. 

FEMA has also had limited success in ensuring that program requirements 
for the National Exercise Program have been followed for Principal Level 
Exercises. 72 HSEEP guidance and the implementation plan state that the 
results of exercises should be documented and corrective action 
responsibilities assigned and tracked. Specifically, HSEEP guidance calls 
for organizations to (1) complete an after-action report with an 
improvement plan and (2) track corrective actions to ensure that they are 
implemented. Based on our review of four after-action reports issued by 



69 TOPOFF 4 cost approximately $22 million to carry out. In addition to TOPOFF 4, two 
Tier II level exercises have been conducted through December 2008. "National Level 
Exercise 2-08" took place in May 2008 and Diablo Bravo took place in July and August 2008. 
As of February 2009, after-action reports and improvement plans had not been completed 
for these two exercises. 

™FEMA officials noted that while corrective actions have been identified from TOPOFF 4, 
their tool to track corrective actions could not be used until the agencies responsible for 
ensuring corrective actions are taken were identified. 

71 According to FEMA officials, DHS could issue the after-action report for TOPOFF 4 later 
this year. 

72 According to the implementation plan for the Exercise Program, HSEEP serves as 
doctrine for the design, development, and conduct of all exercises under the Exercise 
Program, including Tier I Principal Level Exercises. 
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HSC between April 2007 and August 2008, three did not meet program 
requirements. 73 In one case, the after-action report was drafted but not 
finalized. In two other cases, the report did not identify officials who were 
responsible for ensuring corrective actions were implemented. 74 For 
example, the after-action report for the February 2008 Principal Level 
Exercise on pandemic influenza stated that the integration of strategic 
communications and policy remains difficult and should be addressed by 
an interagency group. However, the after-action report did not identify a 
department or an agency official who was to be responsible for ensuring 
that this corrective action was implemented. FEMA officials said they 
relied on the National Exercise Program Implementation Plan as the 
guiding policy for HSC's responsibilities for documenting after-action 
reports and tracking and resolving corrective actions. However, the plan 
does not describe a procedure for ensuring that these requirements are 
met by HSC. Based in part on our review of Principal Level Exercises, 
FEMA has subsequently taken action to identity officials who are 
responsible for nearly all of the corrective actions outlined in these after- 
action reports. 

The implementation plan for the National Exercise Program also requires 
entities responsible for Tier I level exercises, including Principal Level 
Exercises, to ensure that corrective actions are resolved. However, HSC 
did not follow the required corrective action process. According to HSC 
staff, the council does not use the CAP system or another tracking 
procedure for determining whether corrective actions were implemented. 
Rather, it delegates the responsibility of taking corrective actions to the 
appropriate agencies or departments. 

FEMA officials agreed that tracking corrective actions for Principal Level 
Exercises is problematic and impairs their ability to fulfill FEMA's 
statutory obligation to track corrective actions from exercises. 75 However, 
they stated that they do not have the authority to direct HSC or other 



73 We are not reporting on one after-action report because it was classified. 

74 While two of the three after-action reports identified corrective actions, they did not 
always identify the department or agency official who was charged with taking the 
corrective action or identify the milestone date by which corrective actions were to be 
completed. 

75 In March 2009, FEMA officials provided us corrective action program worksheets for 
TOPOFF 4 and a Tier II exercise conducted in May 2008 that have been reviewed by the 
Domestic Readiness Group, the senior policy entity for HSC on these matters. 
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federal entities to track corrective actions and report this information to 
FEMA. FEMA officials said that they rely on the National Exercise 
Program Implementation Plan to set forth HSC's responsibilities for 
documenting after-action reports and tracking and resolving corrective 
actions. 

Although the Post-Katrina Act does not give FEMA the authority to compel 
other federal entities to comply with the objectives of the National 
Exercise Program, the act places responsibility for implementing the 
National Exercise Program on FEMA, in coordination with other 
appropriate federal entities. Therefore, it is incumbent on FEMA to 
coordinate with HSC and other federal entities to better ensure that FEMA 
obtains the information it needs to meet its statutory responsibility to 
track corrective actions. In this regard, the implementation plan for the 
National Exercise Program does not set forth FEMA's statutory 
responsibility to track corrective actions, nor does it require federal 
departments and agencies to report corrective action information to 
FEMA. On the contrary, the implementation plan provides that 
departments and agencies "may submit issues to the DHS Corrective 
Action Program (DHS CAP) through the web-based DHS CAP system," but 
does not instruct them to do so or to otherwise provide FEMA with 
corrective action tracking information that would enable FEMA to fulfill 
its statutory responsibilities under the Post-Katrina Act. Therefore, the 
implementation plan lacks procedures that call on HSC and other federal 
entities to report corrective action information to FEMA. FEMA's inability 
to fully track and analyze areas that need improvement is also due, in part, 
to its lack of an effective internal controls environment. GAO's Standards 
for Internal Controls in the Federal Government state that an effective 
internal control environment is a key method to help agency managers 
achieve program objectives and enhance their ability to address 
weaknesses.™ The standards state, among other things, that agencies 
should have policies and procedures for ensuring that the findings of 
audits and reviews are promptly resolved. The standards also state that 
internal controls should generally be designed to assure that ongoing 
monitoring occurs. Developing procedures for working with federal 
entities, such as HSC, to help ensure that corrective actions are tracked, 
implemented, and reported to FEMA would strengthen FEMA's ability to 
determine emergency management areas that need improvement. 



'GAO/AIMD-00.21.3.1. 
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Lessons learned from Hurricane Katrina identified similar concerns with 
tracking corrective actions from exercises. For example, in February 2006 
the White House report on Hurricane Katrina stated that "too often, after- 
action reports for exercises and real-world incidents highlight the same 
problems that do not get fixed". 77 According to the report, DHS should 
ensure that all federal and state entities are accountable for the timely 
implementation of remedial actions in response to lessons learned. 
According to the report, the success of the preparedness system depends, 
in part, on feedback mechanisms for tracking corrective actions. When 
federal entities carry out processes that are incompatible with FEMA's 
responsibilities for tracking corrective actions, FEMA managers do not 
have the necessary data to measure progress, identify gaps in 
preparedness, and track corrective actions — key objectives of the National 
Exercise Program. 

FEMA Does Not Have Similar to the problems we found with Principal Level Exercises, we 

Procedures to Help Ensure identified weaknesses in the way in which selected states prepared and 

That After- Action Reports from submitted after-action reports to FEMA. 78 Among other things, HSEEP 
State-Level Exercises Are requires that exercise program managers prepare after action-reports that 

Properly Prepared and include improvement plans to identify corrective actions, track whether 

Submitted and That Corrective the actions were implemented, and continually monitor and review 
Actions Are Tracked and corrective actions as part of an organizational corrective action program. 79 

Implemented Exercise program managers are to submit these after-action reports to 

FEMA through its Secure Portal — a FEMA database containing after- 
action reports. Of the six states we visited, (1) none systematically 
recorded or submitted after-action reports to FEMA, (2) only one had 
improvement plans in all of its after-action reports and only one said it had 
a corrective action tracking program, and (3) none used a capabilities- 
based approach in all of their exercises. 80 The following are additional 
details on each of these issues. 



The White House, The Federal Response to Hurricane Katrina: Lessons Learned 
(Washington, D.C.: Feb. 26, 2006). 

78 We interviewed officials from each of the six states we visited and reviewed 44 after- 
action reports/improvement plans that state officials identified to us as using HSGP funds. 

79 FEMA's HSGP guidance for fiscal years 2007 and 2008 states that exercises conducted 
with FEMA support must be managed and executed in accordance with the HSEEP 
guidance. In addition, according to FEMA officials, FEMA's Regional Exercise Support 
Program should ensure that state exercises comply with HSEEP. 

80 In February 2009, officials from one state informed us that they had initiated a corrective 
action program based, in part, on our visit. 
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• HSGP guidance requires exercise program managers to submit after- 
action reports within 60 days following the completion of an exercise 
to FEMA through its Secure Portal. Although the portal has been 
operational for about 5 years, FEMA did not have procedures in place 
to fully monitor state actions and to ensure that this occurred. 81 While 
3 of 44 after-action reports provided by officials from the six states we 
visited were submitted to FEMA in the requisite area of the portal, 
these reports all came from one state. The remaining five states did not 
submit any after-action reports through the portal. Officials from these 
five states cited technical difficulties, lack of staff resources, or unclear 
guidance from FEMA as reasons why after-action reports were not 
submitted to the portal. FEMA is aware that the portal contains 
incomplete information, noting in its quarterly newsletter that "The 
Secure Portal serves as the repository for after action reports and 
improvement plans; however, postings have been inconsistent. 82 At 
times, it has been difficult to locate After Action Reports as many are 
posted in draft form and never finalized and posted outside the 
respective State folder." In February 2009 FEMA announced, a 
National Exercise Division Exercise Support System that is an online 
tool for facilitating exercise planning to replace the Secure Portal as a 
repository for after-action reports. 

• HSEEP requires that entities include an improvement plan as part of 
their after-action reports and that they have a corrective action 
program. While each of the six states we visited had produced at least 
one draft after-action report that included an improvement plan, only 
one state included an improvement plan in all of its reports. Fifteen of 
the 44 after-action reports we reviewed had an improvement plan. In 
addition, only one state had a corrective action program that tracked 
whether corrective actions were implemented. Officials from one state 
attributed the lack of a corrective action program to competing 
priorities. Specifically, states are involved in many exercises and 
officials are more likely to place the priority on designing and 
conducting of the next exercise than on tracking corrective actions 
from prior exercises. 



The FEMA Secure Portal was originally procured in 2002 and fully implemented by June 
2004 to support the mission of the DHS Office for Domestic Preparedness to engage 
federal, state, and local emergency response and preparedness stakeholders in training and 
exercising weapons-of-mass-destruction prevention, protection, response, and recovery. 
Over the years, the mission-support role of this information system has broadened to 
handle the transmission, sharing, and storage of sensitive but unclassified information. 

82 FEMA, National Exercise Division Homeland Security Exercise anal Evaluation 
Program, Quarterly Newsletter (Summer 2008). 
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• Another HSEEP requirement calls for exercises to be designed and 
conducted using a capability-based approach. 83 Doing so would help 
FEMA analyze whether gaps in capability have narrowed and 
improvements in capabilities have occurred from the use of grant funds 
by states. However, 20 of the 44 after-action reports provided by 
officials in the six states we visited used target capabilities, while the 
remaining 24 did not. According to officials from three of the six 
states we visited, not all exercise participants have a good 
understanding of target capabilities and how they should be used in the 
design, conduct, and evaluation of exercises. For example, exercise 
officials from one state said their state does not use target capabilities 
because it has its own set of assessment standards. 

FEMA's lack of procedures for monitoring states to ensure compliance 
with HSEEP requirements contributed to limited adherence to such 
requirements in the states we visited. As discussed earlier in this report, 
internal control standards call for (1) an effective internal control 
environment to help agency managers achieve program objectives and 
enhance their ability to address weaknesses, (2) agencies to have policies 
and procedures for ensuring that the findings of audits and reviews are 
promptly resolved, and (3) internal controls to generally be designed to 
assure that ongoing monitoring occurs. 84 FEMA officials in the National 
Preparedness Directorate told us they have a process for monitoring 
HSEEP compliance by, among other things, having FEMA regional 
exercise support program managers discuss HSEEP compliance with state 
exercise program officials at planning conferences or during grant 
monitoring discussions. While discussing HSEEP requirements at annual 
conferences may enhance the awareness of state officials about 
requirements for HSEEP compliance, this does not track compliance. In 
addition, officials from FEMA's Grant Programs Directorate said they do 
not monitor states' compliance with HSEEP requirements. For example, 
the grant monitoring reports for the six states we visited did not address 
whether the states were in compliance with HSEEP requirements. Such 
reports are based, in part, on a checklist of items that officials use to 
monitor compliance with grant requirements. However, FEMA's checklist 
does not include specific items, such as compliance with HSEEP 
requirements, as called for by HSGP guidance. States' noncompliance 



The after-action report is to include a list of capabilities from the Target Capabilities List 
that were tested and an analysis of participants' effectiveness in executing the capabilities. 

84 GAO/AIMD-00.21.3.1. 
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with HSEEP hinders their ability to systematically track corrective actions 
and assess capabilities. This in turn impacts FEMA's ability to measure 
progress of the National Exercise Program. Having procedures in place to 
monitor actions by states to ensure compliance with HSEEP requirements 
would assist FEMA in obtaining more complete data about the results of 
exercises and corrective actions taken to systematically evaluate readiness 
through the National Exercise Program, as required by the Post-Katrina 
Act. 



FEMA's Ability to Design 
Exercises That Stress the 
Preparedness System Is 
Limited 



The Post-Katrina Act requires FEMA to stress the preparedness system 
through the National Exercise Program to evaluate preparedness for a 
catastrophic event. 85 The National Exercise Program Implementation Plan 
identifies domestic incident management for catastrophic events as the 
principal focus of the National Exercise Program. 86 According to the Tier I 
exercise cycle established in the implementation plan, FEMA plans to test 
for a catastrophic domestic nonterrorism event in fiscal year 20 10. 87 
However, FEMA's ability to meet this testing requirement is limited by 
three factors: (1) the lack of key planning documents, (2) exercise 
artificiality, and (3) limited coordination with groups that have expertise in 
populations with special needs. 



First, the effectiveness of exercises is based, in part, on the degree to 
which plans that define roles and responsibilities have been developed. 



85 Among its Post-Katrina Act responsibilities, at least once every 2 years FEMA is required 
to conduct national exercises for the following purposes: (1) to test and evaluate federal, 
state, local, and tribal capability to detect, disrupt, and prevent catastrophic acts of 
terrorism, and (2) to test and evaluate federal, state, local, and tribal readiness to respond 
to and recover from catastrophic incidents. 6 U.S.C. § 748(b) (3). The Post-Katrina Act 
defines a catastrophic incident as any natural disaster, act of terrorism, or man-made 
disaster that results in extraordinary levels of casualties or damage or disruption severely 
affecting the population, infrastructure, environment, economy, national morale, or 
government functions in an area. 6 U.S.C. § 701(4). 

86 Catastrophic events are different in the severity of the damage, number of persons 
affected, and the scale of preparation and response required. They quickly overwhelm or 
incapacitate local and/or state response capabilities, thus requiring coordinated assistance 
from outside the affected area. Thus, the response and recovery capabilities needed during 
a catastrophic event differ from those required to respond to and recover from a "normal 
disaster." 

87 The Post-Katrina Act appears to require catastrophic testing before this date. Under the 
statute, the exercise program was to be established by April 2007, which would make April 
2009 the deadline for the first 2-year catastrophic testing cycle. See 6 U.S.C. § 748(b)(1), 
(b)(3). 



Page 45 



GAO-09-369 National Preparedness 



The fact that key planning documents for response to a catastrophic 
incident, such as the supplement to the catastrophic incident annex and 
regional response plans, have not yet been completed means that the 
National Exercise Program will have difficulty in designing exercises that 
test whether the plans are understood and executed effectively by 
stakeholders. 88 89 As we described earlier in this report, while DHS and 
FEMA are working on these plans, it is unclear when they will complete 
the plans. According to the former director of FEMA's National 
Preparedness Directorate, FEMA's ability to design and conduct exercises 
that evaluate a response to a catastrophic incident is limited by the fact 
that plans such as those described above have yet to be developed. 
However, the official indicated that FEMA was taking preliminary actions 
to build its capacity to conduct such exercises through regional 
catastrophic planning initiatives. 

Second, in addition to its catastrophic testing requirements, the Post- 
Katrina Act requires that exercises be "as realistic as practicable . . . and 
designed to stress the national preparedness system." 90 An important 
factor limiting FEMA's ability to stress the national preparedness system is 
the difficulty of simulating real-world conditions. All exercises involve 
some degree of artificiality. Two exercises, TOPOFF 4 and a Tier II 
exercise conducted in May 2008, reflected the challenges FEMA faces in 
conducting realistic exercises that stress the preparedness system. For 
example, during TOPOFF 4, the governor's helicopter landed in a 
contaminated area but was not required to undergo the decontamination 
procedures that would have been required in a real-world situation. 
During a May 2008 Tier II exercise, certain scenarios of the exercise, such 
as a response to a terrorist attack involving an industrial chemical release 
(see fig. 5), did not involve the National Operations Center, which would 
have participated in a real-world event. For example, an exercise manager 
with the U.S. Northern Command noted that the nonparticipation of the 
National Operations Center was a limiting factor in testing roles and 
responsibilities with other federal entities. According to the National 



88 The supplement to the catastrophic incident annex is required by the Post-Katrina Act. 6 
U.S.C. § 319(b)(2)(C). While a version of the supplement was written before the act, an 
updated supplement has yet to be published. 

89 Regional plans have been drafted; however, according to FEMA officials, the plans are not 
consistent from one region to another because regions developed plans without any 
guidance from FEMA headquarters. 

90 6 U.S.C. § 748(b)(2)(A)(i). 
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Exercise Program's Implementation Plan, such participation from other 
federal stakeholders is not required for Tier II exercises such as the one 
held in May 2008. 



Figure 5: Exercise Participants Observing a Federal, State, and Local Response to 
a Terrorist Attack Involving a Chemical Release during a May 2008 Tier II Exercise 




Source: GAO. 



Third, another challenge in creating exercises that stress the preparedness 
system and simulate real-world conditions is finding ways to test response 
capabilities for populations with special needs. To address some of the 
problems experienced in Hurricane Katrina in dealing with populations 
with special needs, such as residents in nursing homes, the Post-Katrina 
Act, as amended by the 9/11 Act, called on FEMA to design exercises to 
address the unique requirements of populations with special needs, 
including the elderly, and to coordinate the National Exercise Program 
with the National Council on Disability, among other entities. 91 In 
TOPOFF 4, FEMA integrated specific objectives for special needs 
populations in the Oregon venue, according to FEMA officials. For 



6U.S.C. § 748(b)(1), (b)(2)(A)(v). 
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example, according to FEMA officials, FEMA utilized special needs actors 
to enhance realism. However, HSEEP guidance does not address special 
needs populations. Further, while FEMA has corresponded with the 
Council on Disability, council officials believe that FEMA could do more 
to ensure that exercises are designed to address the unique requirements 
of populations with special needs. For example, council officials stated 
that the council was not involved in the design and planning for TOPOFF 4 
or the May 2008 Tier II exercise. According to officials from the National 
Preparedness Directorate, it coordinated with FEMA's Special Needs 
Office to integrate special needs population objectives into exercises. 
FEMA officials agree that special needs populations should be included in 
exercises and they said that they will redouble their efforts to do so. 
However, FEMA officials also noted that some exercises, for example, the 
National Level Exercise planned in July 2009, may not involve special 
needs populations because the point of such exercises is to prevent a 
terrorist attack, rather than to test response and recovery efforts. 
Enhancing coordination with the National Council on Disability could 
improve FEMA's ability to ensure that key issues concerning populations 
with special needs are addressed in the design and conduct of exercises. 

The limitations the National Exercise Program faces in designing 
approaches that stress the preparedness system highlight the difficulty in 
validating whether roles and responsibilities are well understood and 
major gaps in capabilities remain for responding to and recovering from 
catastrophic events. The 2006 White House report on the federal response 
to Katrina concluded that the "national preparedness system must be 
oriented toward greater challenges. We must not shy away from creating 
scenarios that stress the current system of response to the breaking 

point Until we meet the standards set by the most demanding 

scenarios, we should not consider ourselves adequately prepared." 92 In 
2006, we reported that effective exercises should involve scenarios that 
stress responders with the highest degree of realism possible, even to the 
breaking point if possible. 93 Exercises that stress the preparedness system 
in a realistic way are key to testing the prospective reliability of a response 



"The White House, The Federal Response to Hurricane Katrina: Lessons Learned 
(Washington D.C.: Feb. 26, 2006). 

93 GAO, Hurricane Katrina: Better Plans and Exercises Needed to Guide the Military's 
Response to Catastrophic Natural Disasters, GAO-06-643 (Washington, D.C.: May 15, 
2006). 
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and determining whether plans have accounted for potential breakdowns 
with relatively greater consequences. 

In February 2009, we met with FEMA officials to discuss this issue and 
they agreed that developing exercises to the point of system failure is a 
valid objective; however, they described several factors that may limit 
their ability to do so. For example, FEMA officials told us that exercising 
to the "breaking point" requires significant resources that under current, 
and likely future funding streams, are unlikely to be available. We agree 
with FEMA that these are important considerations; however, these 
considerations, in part, are addressed through the implementation plan for 
the National Exercise Program, which describes a 5-year schedule of 
exercises to give federal departments and agencies lead time to budget for 
participation in such events. 94 



FEMA's Data for Measuring The three databases that FEMA uses to measure the effectiveness and 
the Effectiveness Of Its progress of the National Exercise Program have incomplete data. FEMA 

National Exercise Program uses ^ ^ e NEXS system to identify DHS-funded exercises, (2) the FEMA 
Is Incomplete Secure Portal as a repository for DHS funded exercise after-action reports, 

" and (3) the CAP system as a tool for tracking corrective actions. 95 The 

following provides details on problems with the reliability of each of these 
databases. 



• FEMA calls on state entities to use the NEXS system to schedule all 
exercises, and one of the performance measures that FEMA uses to 
assess and report on the performance of the National Exercise 
Program is the number of DHS-funded state exercises that occur per 
year. However, 26 of the 44 after-action reports we reviewed did not 
have the exercise entered into the NEXS system. Incomplete NEXS 
system data limit FEMA's ability to accurately report on the number of 
DHS-funded exercises. Furthermore, while FEMA created the NEXS 
system to schedule, synchronize, and avoid conflicts in all national, 
federal, state, and local exercises, it cannot do so with incomplete data. 
When we discussed this problem with state and FEMA officials, they 
agreed that the NEXS system did not contain a comprehensive list of all 
state and local exercises supported by HSGP funds. For example, 



94 The National Exercise Program Five-Year Exercise Schedule indicates that a National 
Level Exercise is planned for May 2011 that will focus on a major earthquake. 

95 According to FEMA, exercise planners should post national, federal, state, and local-level 
exercises to the NEXS system. 
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when we asked FEMA for a complete list of all exercises to be 
conducted under the National Exercise Program, the agency could not 
produce such a list. Standards for Internal Control in the Federal 
Government state that program managers need data to determine 
whether they are meeting their performance targets and that controls 
should be designed to validate the integrity of organizational 
performance measures and indicators. 96 FEMA has initiated actions to 
validate the accuracy of data used in the NEXS system by using training 
and exercise plan workshops with states to determine what exercises 
states have scheduled. We agree that the training and exercise plan 
workshops are a good starting point for verifying the completeness of 
NEXS; however, when we attended a training and exercise plan 
workshop, we were told by FEMA officials that not all federal agencies 
or local entities participated, thus, all exercises were not discussed at 
the workshop. In addition, FEMA officials recognize that the 
workshops alone do not ensure that the NEXS database is complete 
and accurate. In the absence of systematic and comprehensive 
information on the number of federally funded exercises, FEMA cannot 
measure its progress in implementing the National Exercise Program. 
• A second database used by FEMA is the Secure Portal — the primary 
database that FEMA uses to measure the degree to which states 
comply with HSEEP. Even though FEMA requires state exercise 
program managers to place their after-action reports in the Secure 
Portal when federal grant funds are used to support the exercise, this 
requirement was not completely met by any of the six states we visited. 
In addition, although state and FEMA officials agreed that the Secure 
Portal does not contain all state exercise after-action reports, FEMA 
uses information from the portal to assess and report on the 
performance of the National Exercise Program, including the measure 
that FEMA uses to assess the percent of DHS-funded exercises 
demonstrating the use of HSEEP guidance. 97 Since the Secure Portal 
contains incomplete information and neither FEMA nor the six states 
we visited have controls to ensure that all state-level exercise after- 



Jb GAO/AIMD-00-21.3.1 

9 This performance measure addresses one element of HSEEP guidance, i.e., that exercises 
must be capability based. FEMA reviews after-action reports to determine whether they 
follow HSEEP guidance on exercise design, and evaluation and considers after-action 
reports that include an analysis of target capability performance to be in compliance with 
minimum HSEEP guidance standards. However other aspects of HSEEP guidance — such as 
whether the after-action report included an improvement plan that identified specific 
corrective actions that must be taken to remedy issues observed during exercise 
evaluation — are not addressed as part of this performance measure. 
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action reports are uploaded to the Secure Portal, this measure may not 
accurately reflect the percent of DHS-funded exercises that 
demonstrate the use of HSEEP guidance. 
• Third, although federal entities involved in Tier 1 exercises are 
encouraged to use FEMA's CAP system, it does not contain all 
corrective actions from such entities. According to FEMA officials, the 
CAP system was designed to capture all relevant and necessary 
information related to the implementation of corrective actions. 
However, the HSC did not use the CAP system to track corrective 
actions. The problem with having incomplete data in the CAP system 
is that FEMA uses information from the system to measure the 
percentage of corrective actions that have been implemented as one of 
its performance measures. 98 A key reason for this problem is that 
FEMA has not established procedures to ensure that the information in 
the CAP system is complete. For example, the implementation plan 
does not require the nine federal departments and agencies that are 
signatories to the implementation plan to use the CAP system. Instead, 
FEMA strongly urges stakeholders to use the system, but the decision 
to do so is discretionary. FEMA officials cited the tension between 
requiring entities to use the CAP system and providing enough 
flexibility to those entities to carry out their programs as a reason for 
not making the use of the CAP system a requirement. Nonetheless, 
entities could submit a report to FEMA on the status of their corrective 
actions resulting from such exercises. 

Finally, the CAP system does not include corrective actions from real- 
world incidents and FEMA has not established requirements or guidelines 
for agencies to do so. As a result, FEMA is unable to meet Post-Katrina 
Act requirements for conducting long-term trend analyses of corrective 
actions that include real-world events. The Presidential Inaugural 
Ceremony held in Washington D.C. on January 20, 2009, provides an 
example of the importance of tracking corrective actions for real- world 
events. During the event, problems with managing crowds prevented a 
large number of ticket holders from reaching their designated area to 
observe the inauguration ceremony. According to the Joint Congressional 
Committee on Inaugural Ceremonies, a complete examination will take 
place to provide a foundation of lessons learned for future inaugural 
planners, so that they can avoid similar problems in the future." FEMA 



98 The NEP Implementation Plan provides for the use of the CAP system for NEP Tier I 
exercises with respect to corrective actions for unclassified issues. 

"joint Congressional Committee on Inaugural Ceremonies: News Release (January 22, 
2009). 
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officials agreed that the CAP system could be used to track corrective 
actions from real-world events, and in February 2009 they indicated that 
developing procedures to do so would aid their ability to conduct long- 
term trend analyses of real-world events as required by the Post-Katrina 
Act. 



FEMA Has Taken 
Actions to Develop 
and Implement a 
Comprehensive 
System for Assessing 
National 
Preparedness 
Capabilities, but 
Faces Challenges in 
Completing and 
Implementing the 
System 



FEMA has taken initial actions to collect information on state 
preparedness capabilities and develop a comprehensive assessment 
system for assessing capabilities at all levels of government, but faces 
methodological and coordination challenges in completing the system. 
Assessing and reporting on national preparedness is a long-standing and 
complex effort that presents methodological, integration, and coordination 
challenges. Effectively addressing these challenges requires that FEMA 
take a measured and planned approach; however, FEMA's project 
management plan does not fully identify the numerous program elements 
and how and when they will be developed and integrated. 
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FEMA Has Taken Initial 
Actions to Develop a 
Comprehensive 
Assessment System and 
Collect Information for 
State and Federal 
Capability Reporting 



The Post-Katrina Act requires that FEMA establish a comprehensive 
assessment system to assess the nation's capabilities and overall 
preparedness for preventing, responding to, and recovering from natural 
and man-made disasters. 100 The act also requires that FEMA collect 
information on state capability levels and report on federal preparedness 
to Congress, including, among other things, the results of the 
comprehensive assessment system. 101 In response to these requirements, 
FEMA established guidance for reporting on state preparedness and 
created the Office of Preparedness Policy, Planning and Analysis (PPPA) 
to develop and implement a new assessment approach that considers past 
efforts and integrates ongoing assessment efforts. 102 FEMA plans to 
integrate the state preparedness reports — along with a variety of existing 
assessment efforts and data sources — into the new comprehensive system 
it is establishing. In addition, it is considering the historical experiences 
and lessons learned from prior assessment efforts in developing the new 
system. 

FEMA has also made progress in collecting information for federal and 
state reporting. In January 2009, FEMA issued its first federal 
preparedness report. The Post-Katrina Act requires that FEMA, in 
coordination with the heads of appropriate federal agencies, submit a 
federal preparedness report to Congress beginning in October 2007 and 
annually thereafter, which is to include, among other things, the results of 
the comprehensive assessment. 103 In addition, states, territories, and the 



The Post-Katrina Act requires FEMA, in coordination with the National Council on 
Disability and the National Advisory Council, to establish a "comprehensive system to 
assess, on an ongoing basis, the Nation's prevention capabilities and overall preparedness, 
including operational readiness." The assessment system must assess (i) compliance with 
the national preparedness system, National Incident Management System, National 
Response Plan (now known as the National Response Framework), and other related plans 
and strategies; (ii) current capability levels as compared to target capability levels; (iii) 
resource needs to meet target capability levels; and (iv) the performance of training, 
exercises, and operations. 6 U.S.C. § 749. 

101 The Post-Katrina Act requires that FEMA submit a federal preparedness report to 
Congress in October 2007 and annually thereafter. The act also requires the submission of 
annual state preparedness reports to FEMA, beginning January 2008, by recipients of DHS 
preparedness assistance, including states, territories, or the District of Columbia. 6 U.S.C. § 
752(a), (c); see also 6 U.S.C. §§ 101(15), 701(11) for the applicable definition of a "state." 

102 In January 2008, FEMA established the Office of Preparedness Policy, Planning and 
Analysis within the National Preparedness Directorate to conduct, among other things, 
capability assessments and analysis of policy and program results assessment efforts to 
develop the comprehensive assessment system. 

103 6 U.S.C. § 752(a). 
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District of Columbia completed and submitted their first state 
preparedness reports to FEMA in the spring of 2008 — a total of 56 reports 
from the 56 jurisdictions receiving homeland security grant funding. 
FEMA officials said they prepared summaries of the 56 reports and 
provided the summaries to FEMA's regional offices. In addition, in 
November 2008, FEMA issued guidance for the 2008 state preparedness 
reports, which grantees are to submit to FEMA by March 2009. 



FEMA Faces 
Methodological and 
Coordination Challenges in 
Completing the 
Comprehensive 
Assessment System and 
Reporting on Its Results 



Methodological Challenges 
with Using Four Proposed 
Assessment Approaches 



FEMA faces methodological challenges with the four assessment systems 
it plans to use as the basis for the new system and has not determined how 
to overcome problems faced in historical assessments. The challenges 
FEMA faces reflect the lack of guidance from PPPA in how the assessment 
system will comprehensively inform and incorporate feedback from other 
elements of the National Preparedness System and information from a 
variety of other data sources. Finally, FEMA faces challenges in 
coordinating with state, local, and federal stakeholders in developing and 
implementing the system and reporting on its results. 

In December 2008, FEMA provided us with a project management plan 
outlining efforts to establish the comprehensive assessment system by 
May 2010 to "function as a central repository for national preparedness 
data." The system "will integrate data from prior reports and legacy 
assessment systems." To establish the system, FEMA plans to administer a 
Web-based survey to all states and territories in the summer of 2009 to 
assess capabilities using the 37 target capabilities. FEMA plans to use a 
Web-based system known as the National Incident Management System 
Compliance Assessment Support Tool (NIMSCAST) to administer the 
capability assessment survey. In addition, FEMA noted that NIMSCAST 
will serve as the technical foundation for the comprehensive assessment 
system, and that the system is used by all states and territories as well as 
by 18,000 local and tribal entities, which helps to mitigate challenges 
FEMA faces in coordinating with stakeholders in developing and 
implementing the comprehensive assessment system. However, FEMA 
faces methodological challenges with regard to (1) differences in data 
available, (2) variations in reporting structures across states, and (3) 
variations in the level of detail within data sources requiring subjective 
interpretation, as summarized in table 2 below. Additional information 
regarding these assessments is outlined in appendix III. 
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Table 2: Summary of Assessment Approaches 



Assessment/FEMA Directorate 



Limitations 



State Preparedness Reports 

Led by FEMA's National Preparedness 
Directorate 



FEMA could not use information in the 2007 reports to compare capability gaps 
between states because reports are in narrative format and information is not based 
on common metrics for assessing capabilities. 



National Incident Management System 
Compliance Assessment Support Tool 
(NIMSCAST) 

Led by FEMA's National Preparedness 
Directorate 



NIMSCAST assesses states' and territories' compliance with the National Incident 
Management System, which is a standardized process for conducting response 
operations, rather than a method for assessing capabilities. As a result, it is unclear 
how FEMA will integrate NIMSCAST with efforts related to target capabilities, as it 
plans to do. 



Gap Analysis Program (GAP) 

Led by FEMA's Disaster Operations 
Directorate 



Provides information related to 7 of 37 target capabilities, such as sheltering or debris 
removal, but does not include comprehensive information for activities related to all 37 
target capabilities. 



Cost to Capability Initiative 

Led by FEMA's Grant Programs Directorate 



Grantees will not be required to submit information to the Cost to Capability Initiative, 
the purpose of which is to help FEMA, states, and localities better measure the results 
of federal grants. Thus FEMA cannot have assurance that it will collect 
comprehensive information to assess how grant funds have improved capabilities. 



Source: GAO analysis of DHS and FEMA information. 



Methodological Challenges m addition to the methodological challenges in its current approach to 

Identified in Historical assessing capabilities, previous efforts at assessing capabilities 

Assessment Approaches experienced challenges and have been discontinued. 

• The National Preparedness System was discontinued by DHS officials 
because it was time-consuming and did not produce meaningful data, 
according to FEMA officials. The system was pilot tested in 10 states 
and according to budget documentation, FEMA spent nearly $15 
million in total for 2006, 2007, and 2008 on the system before it was 
discontinued. 104 

• The Pilot Capability Assessment was labor-intensive and did not 
generate meaningful data, according to FEMA officials. This 
assessment, piloted in six states, was intended to measure 
jurisdictions' progress in achieving needed target capabilities. 105 
Because it was only piloted, FEMA did not generate meaningful 



The National Preparedness System was piloted in Alabama, California, Colorado, Idaho, 
Iowa, Maryland, New Hampshire, New Jersey, Rhode Island, and Virginia. The system is a 
stand-alone Web-based system, and is different from the National Preparedness System 
that is required by the Post-Katrina Act to ensure that the nation has the ability to deal with 
all-hazards incidents. 

105 The Pilot Capability Assessment was pilot tested in Colorado, Florida, Michigan, 
Minnesota, Pennsylvania, and Utah. 
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preparedness information from the data collected, according to 
officials. 

• The Capability Assessment for ReadinesSi which was proposed as a 
one-time nationwide assessment of capabilities, lacked controls for 
validating the accuracy of self-reported assessment data. The 
assessment was conducted in 1997 but concerns regarding self- 
reporting and the lack of controls for validating information reported 
by states limited the reliability and, therefore, the value of the data, 
according to the DHS Inspector General. 

Additional information regarding these efforts is outlined in appendix III. 



FEMA Faces Integration FEMA has not established an approach for how information and data from 

Challenges in Completing the different sources will be integrated into the comprehensive assessment 
Comprehensive Assessment system. FEMA officials have established a charter between the National 
System Preparedness Directorate and Grant Programs Directorate to coordinate 

preparedness efforts related to the Cost to Capability Initiative and 
refinement of the target capabilities. FEMA has also begun sharing 
information between staff involved in developing the assessment system 
and staff involved in other elements of the National Preparedness System. 
For example, staff from the National Exercise Program said they shared 
information on their exercise efforts with staff developing the 
comprehensive assessment system and FEMA officials said they 
established a working group of officials from other federal agencies to 
communicate efforts to develop the comprehensive assessment system. In 
March 2009, FEMA officials acknowledged that they had not finalized a 
charter for this working group to outline the specific actions that the 
working group will undertake to develop the comprehensive assessment 
system. FEMA explained that this working group will (1) identify existing 
sources of data related to preparedness plans, organization, equipment, 
training, and exercises; (2) vet the relevancy of each data source for 
assessments; (3) identify data gaps and redundancies; and, (4) develop 
recommendations for streamlining data collection and reporting. 
However, FEMA has not established an approach for integrating 
information and data from other stakeholders, including grantees and 
other FEMA divisions such as the Disaster Operations Directorate. 

In October 2008, FEMA officials said they also plan to consider or 
incorporate into the new system a multitude of other data and analysis 
sources within and outside of DHS, such as FEMA's Biannual Strategy 
Implementation Reports; Homeland Security Grant Program Investment 
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Justification; and Tactical Interoperable Communications Plan 
Scorecards. 106 In addition, FEMA plans to use the CAP system and LLIS to 
inform the comprehensive assessment system. In February 2009, FEMA 
officials further explained that they plan to rely on three indicators of 
preparedness to develop the comprehensive assessment system: (1) state 
and federal preparedness reports, which are required to use target 
capabilities; (2) the results of exercise corrective action findings; and (3) 
operational plans outlining specific operational requirements for all levels 
of government. However, FEMA has not established an approach for how 
the three indicators of preparedness will be collected and developed into 
reporting mechanisms that meet Post-Katrina Act requirements for the 
comprehensive assessment system. In its first federal preparedness 
report, FEMA acknowledged that its efforts to evaluate and improve 
preparedness are the least mature elements of the national preparedness 
system because these efforts are composed of a wide range of systems and 
approaches with varying levels of integration. Given the relative 
immaturity of FEMA's evaluation and improvement efforts, without an 
approach for integrating its comprehensive assessment system efforts, 
FEMA faces increased risks that inconsistencies may occur or that data 
and information are not shared, limiting FEMA's ability to fulfill the 
requirements of the Post-Katrina Act for developing the system. 



FEMA Faces Coordination 
Challenges in Establishing 
Quantifiable Metrics and 
Collecting Information for 
Implementing the 
Comprehensive Assessment 
System 



In addition to methodological and coordination challenges in developing 
and completing the comprehensive assessment system, FEMA faces 
coordination challenges in establishing quantifiable metrics for target 
capabilities outlined in the Target Capabilities List. Establishing 
quantifiable metrics for target capabilities is a prerequisite to developing 
assessment data that can be compared across all levels of government. 107 
At the time of our review, FEMA was in the process of refining the target 
capabilities to make them more measurable and to provide local and state 



Awardees of the Homeland Security Grant Program (HSGP) are required to provide 
FEMA with semiannual updates on obligation and expenditure information through 
Biannual Strategy Implementation Reports. Applicants to the HSGP are required to provide 
an HSPG Investment Justification that addresses each initiative being proposed for 
funding. Tactical Interoperable Communications Plan Scorecards assess the maturity of 
interoperable communications capabilities in urban and metropolitan areas. 

107 The Post-Katrina Act requires FEMA, in developing guidelines to define target 
capabilities, to ensure that such guidelines are specific, flexible, and measurable. In 
addition, FEMA must ensure that each component of the national preparedness system, 
which includes the target capabilities, is developed, revised, and updated with clear and 
quantifiable performance metrics, measures, and outcomes. 6 U.S.C. §§ 744(b)(1), 746(c), 
749(b). 
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jurisdictions with additional guidance on the levels of capability they need. 
FEMA plans to develop quantifiable metrics — or performance objectives — 
for each of the 37 target capabilities that are to outline specific capability 
"targets" that jurisdictions, including but not limited to cities, of varying 
size should strive to meet. FEMA plans to complete quantifiable metrics 
for all 37 target capabilities by the end of 2010. As of February 2009, 
FEMA noted that 6 of the 37 capabilities were undergoing stakeholder 
review and that FEMA planned to develop quantifiable metrics for a total 
of 12 capabilities during the 2009 calendar year. However, as of March 
2008, FEMA had not developed milestones for completing quantifiable 
metrics for the remaining 25 target capabilities. Cognizant of the fact that 
there is not a "one size fits all" approach to preparedness, FEMA also plans 
to develop performance classes for each target capability in order to 
account for differences in levels of preparedness across jurisdictions of 
varying size and risk. FEMA plans to incorporate the new performance 
objectives and performance classes into the comprehensive assessment 
system, the federal preparedness report, and the guidance for the state 
preparedness reports, but has not established a time frame for doing this. 

FEMA recognizes the need to coordinate with federal, state, and local 
stakeholders and ensure that their views are effectively integrated in the 
development of the metrics, but FEMA historically has faced challenges in 
coordinating with stakeholders. For example, as we reported in June 
2008, FEMA's efforts to coordinate with stakeholders in developing the 
NRF were inconsistent and needed to be improved. 108 In addition, such 
coordination can be time consuming. For example, in July 2005 we 
reported on DHS's prior effort to develop a tiered system of metrics based 
on population density and critical infrastructure in order to (1) assign 
jurisdictions responsibility for developing and maintaining target 
capability levels and (2) use the metrics to implement a national "balanced 
investment program" (with the purpose of directing federal preparedness 
assistance to the highest priority capability gaps) for national 
preparedness capabilities. DHS scheduled this system to be developed 
and completed by October 2008. 109 At the time of our review, these efforts 
to develop quantifiable metrics for target capabilities were not complete, 



GAO, National Response Framework: FEMA Needs Policies and Procedures to Better 
Integrate Non-Federal Stakeholders in the Revision Process, GAO-08-768 (Washington, 
D.C.: June 11, 2008). 

109 GAO, Homeland Security: DHS' Efforts to Enhance First Responders' All-Hazards 
Capabilities Continue to Evolve, GAO-05-652 (Washington, D.C.: July 11, 2005). 
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illustrating the fact that developing metrics in coordination with a variety 
of stakeholders can take longer than anticipated. In developing the 
quantifiable capability metrics, FEMA officials told us that they plan to 
conduct extensive coordination with stakeholders that will entail 
conducting stakeholder workshops in all 10 FEMA regions and 
coordinating with all federal agencies with lead and supporting 
responsibility for Emergency Support Function (ESF) activities associated 
with each of the 37 target capabilities. 110 111 Officials said they also plan on 
briefing the National Advisory Council and the National Council on 
Disability, and soliciting public comment on the draft quantifiable metrics 
for each target capability. 

One of FEMA's coordination efforts — working with nonfederal 
stakeholders and federal agencies responsible for ESF activities — 
illustrates the large number of stakeholders with whom FEMA plans to 
coordinate in developing quantifiable metrics for the target capabilities. 
FEMA also plans to post each revised capability to the Federal Register 
for comment. 112 With respect to federal agency coordination, FEMA plans 
to coordinate with each federal agency that has lead and supporting 
responsibility for ESF activities associated with each of the 37 target 
capabilities in developing quantifiable capability metrics. 113 For example, 



The Emergency Support Function (ESF) structure of the National Response Framework 
provides the mechanism for coordinating federal interagency support for a federal 
response to an incident, and groups functions most frequently used to provide federal 
support to states and federal-to-federal support during a disaster. The National Response 
Framework includes 15 ESFs ranging from transportation to external affairs issues, with 
federal agencies designated as a coordinator, primary agency, or supporting agency (e.g., 
Department of Transportation is the coordinator and primary agency for ESF #1 
Transportation). Each of the 37 target capabilities outlined in the Target Capabilities List 
is assigned one or more ESF. 

m FEMA does not plan to develop quantifiable metrics needed to assess capabilities at the 
FEMA regional level, despite agency efforts to focus on and enhance regional preparedness 
in response to Post-Katrina Act requirements. While the act does not specifically require 
the development of quantifiable metrics to assess capabilities at the regional level, it does 
require capability building at the regional level. For example, the act requires each FEMA 
region to assist in the development of regional capabilities needed for a national 
catastrophic response system, and to identify critical gaps in regional capabilities to 
respond to populations with special needs. 6 U.S.C. § 317(c)(2)(B), (G). 

112 Published by the Office of the Federal Register, National Archives and Records 
Administration (NARA), the Federal Register is the official daily publication for rules, 
proposed rules, and notices of federal agencies and organizations, as well as executive 
orders and other presidential documents. 

113 Each of the 37 target capabilities is assigned to one or more federal agencies that serve 
as coordinating, primary, or support agencies for conducting ESF activities. 
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for the medical surge capability, the Department of Health and Human 
Services (HHS) is the primary federal agency responsible for coordinating 
necessary medical surge capabilities needed during a disaster to provide 
triage and medical care services. In addition to HHS, 15 other federal 
agencies and the American Red Cross are designated as supporting 
agencies and organizations for medical surge capabilities. Coordinating 
with other federal agencies responsible for ESF activities in developing 
quantifiable capability metrics would likely entail time, effort, and 
unforeseen risks. For example, in September 2008 we reported on the 
risks that FEMA faces in coordinating with external stakeholders, namely 
the American Red Cross, in collecting and integrating preparedness 
information necessary to develop the comprehensive assessment 
system. 114 In that report, we recommended that FEMA take steps to better 
incorporate information from voluntary organizations related to sheltering 
and feeding capabilities, which are elements of the mass care target 
capability, and noted that a comprehensive assessment of the nation's 
capabilities should account as fully as possible for the voluntary 
organizations' capabilities in mass care. FEMA disagreed with the 
recommendation, noting that it cannot control the resources of nonprofit 
and private organizations. In response, we (1) noted that taking steps to 
assess capabilities more fully does not require controlling these resources, 
but rather cooperatively obtaining and sharing information and (2) 
reiterated that such efforts are important for assessing the nation's 
prevention capabilities and overall preparedness. 

FEMA's efforts to collect information needed to draft and issue the first 
federal preparedness report, required by the Post-Katrina Act, also reflect 
the coordination challenges FEMA faces in implementing the 
comprehensive assessment system. The Post-Katrina Act requires that 
FEMA, in coordination with the heads of appropriate federal agencies, 
submit a federal preparedness report to Congress beginning in October 
2007 and annually thereafter, which is to include, among other things, the 



GAO, Voluntary Organizations: FEMA Should More Fully Assess Organization's Mass 
Care Capabilities and Update the Red Cross in Catastrophic Events, GAO-08-823 
(Washington, D.C.: Sept. 18, 2008). In March 2008, the DHS Inspector General also reported 
on the difficulties FEMA faces in coordinating with stakeholders to assess capabilities. The 
Inspector General reported that FEMA had made modest progress in assessing capabilities 
at all levels of government and reported that conducting nationwide assessments will 
require coordination among all government levels and the private sector. See Department 
of Homeland Security, Office of Inspector General, FEMA's Preparedness for the Next 
Catastrophic Disaster, OIG-08-34 (Washington, D.C.: Mar. 28, 2008). 
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results of the comprehensive assessment. 115 FEMA issued the first federal 
preparedness report in January 2009. In response to our comment that the 
draft report had been under review for 8 months between March 2008 and 
November 2008, FEMA noted that after completing a review of the report 
by FEMA and DHS, the report was submitted to the Office of Management 
and Budget to disseminate to all federal departments and agencies for 
review and comment. Officials explained that in developing the report, 
they faced challenges in obtaining information and data from federal 
agencies because of bureaucratic obstacles for collecting information and 
also faced challenges in analyzing information from multiple sources. 
FEMA officials said they may develop a National Preparedness Report to 
combine two Post-Katrina Act reporting requirements — the requirement 
for an annual federal preparedness report and an annual catastrophic 
resources report — and to include information from state preparedness 
reports as part of this consolidated report, which they tentatively plan to 
issue in the Spring of 2009. 116 



Despite the methodological and coordination challenges associated with 
developing a new comprehensive assessment system and establishing 
related quantifiable metrics for target capabilities, FEMA has not 
developed an approach that addresses program risks as part of its project 
management plan for how it will develop the comprehensive assessment 
system. While FEMA has developed a project management plan for 
completing the comprehensive assessment system by 2010, the lack of (1) 
milestones for establishing quantifiable metrics for all 37 target 
capabilities and (2) specific actions for how FEMA will integrate 
preparedness information to develop the system, coupled with the (3) the 
lack of risk assessment information for the system raises questions about 
FEMA's ability to establish the system in accordance with its anticipated 
2010 completion date. FEMA has described several steps for completing 
an effective comprehensive assessment system that include developing 
methodologies to translate the information from the assessments FEMA 
has identified into a target capabilities-based framework and integrating 
necessary preparedness information from disparate and not necessarily 
comparable sources such as state preparedness reports. FEMA also plans 
to coordinate with relevant stakeholders to refine the target capabilities. 



'6 U.S.C. § 752(a). 
'6 U.S.C. § 752(a)-(b). 



FEMA Has Not Developed 
an Approach and Assessed 
Risks for Completing and 
Implementing the 
Comprehensive 
Assessment System as Part 
of Its Project Management 
Plan 
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However, FEMA's plan does not outline specific actions it plans to take to 
do so. 

Certain factors, such as challenges in coordinating with stakeholders or 
difficulties in obtaining necessary data, could affect FEMA's ability to 
implement the comprehensive assessment system. Best practices for 
project management established by the Project Management Institute state 
that managing a project involves project risk management, which serves to 
increase the probability and impact of positive events, and decrease the 
probability and impact of events adverse to the project. Project risk 
management entails determining which risks might affect a project, 
prioritizing risks for further analysis by assessing their probability of 
occurrence, and developing actions to reduce threats to the project. Other 
practices include (1) establishing clear and achievable objectives; (2) 
balancing the competing demands for quality, scope, time, and cost; (3) 
adapting the specifications, plans, and approach to the different concerns 
and expectations of the various stakeholders involved in the project; and 
(4) developing milestone dates to identify points throughout the project to 
reassess efforts underway to determine whether project changes are 
necessary. 117 FEMA has demonstrated its awareness of the value of these 
practices. For example, in planning another project — an effort to 
transition the FEMA Secure Portal to an alternate information technology 
platform known as the National Exercise Division Exercise Support 
System — FEMA identified key elements such as phases, milestones, and 
risks that could affect the project goals. 118 

Furthermore, a risk assessment could help FEMA define the specific 
actions to take to complete the comprehensive assessment system, 
anticipate potential delays in completing its efforts to refine the target 
capabilities by 2010, and deal with the associated risks in its efforts to do 
so, such as the time it takes to coordinate with stakeholders. Information 
from a risk assessment could also enhance FEMA's ability to coordinate 
with federal agencies to obtain preparedness information needed to 
produce a timely annual federal preparedness report and catastrophic 
resources report. 119 Until FEMA assesses ways to mitigate the risks 



Project Management Institute, A Guide to the Project Management Body of Knowledge 
(PMBOK Guide), 4th ed. (Newton Square, Pa.: 2008). 

118 FEMA, FEMA Secure Portal Transition Plan (Washington, D.C.: Oct. 15, 2008). 
119 6 U.S.C. § 752(a)-(b). 
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associated with its capability assessment efforts, it will be difficult for 
FEMA to provide reasonable assurance that it can produce a 
comprehensive assessment system that (1) fulfills the requirements of the 
Post-Katrina Act and in the long term, (2) informs decisions related to 
improving national preparedness. 



FEMA's National Preparedness Directorate does not have a strategic plan 
for implementing the national preparedness system. The complexity and 
difficulty of implementing the national preparedness system, which we 
describe earlier in this report, underscore the importance of strategic 
planning. The six desirable characteristics of a national strategy can help 
the National Preparedness Directorate in developing a strategic plan. 

by a Strategic Plan 
That Includes 
Desirable 
Characteristics 



FEMA's 

Implementation of the 
National 

Preparedness System 
Would Be Enhanced 



FEMA's National 
Preparedness Directorate 
Does Not Have a Strategic 
Plan for Implementing the 
National Preparedness 
System 



While FEMA has recognized that its components need to develop strategic 
plans that detail program goals, objectives, and strategies, FEMA's 
National Preparedness Directorate (Preparedness Directorate) has not yet 
developed such a plan for the national preparedness system. 120 In January 
2008, FEMA issued its agencywide strategic plan, which set a common 
direction for its components in carrying out their responsibilities in 
preparedness, response, and recovery programs. 121 122 While a 
Preparedness Directorate official acknowledged that the Preparedness 



FEMA components include disaster operations, grant programs, continuity programs, 
and national preparedness. 

121 FEMA, FEMA Strategic Plan: Fiscal Years 2008-2013 (Washington, D.C.: January 2008). 

122 For example, in October 2008 FEMA's grants directorate issued its component strategic 
plan. The plan states that the cornerstone of the grant directorate's future lies in its 
strategic planning effort. Among other things, the plan recognizes the importance of 
improving grants management by developing a highly trained workforce to carry out the 
grants program, establishing metrics for timeliness and responsiveness to stakeholders, 
and increasing efficiencies across various grants programs by integrating processes across 
programs. 
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Directorate does not have a strategic plan (this is the responsibility of 
FEMA's Office of Preparedness Policy, Planning and Analysis), the official 
said the Post-Katrina Act provides a roadmap that contains the 
preparedness strategy and FEMA uses an annual operating plan (in draft 
form at the time of our review) to guide the directorate's approach for 
implementing the national preparedness system. 123 

Although the Post-Katrina Act and the directorate's draft annual operating 
plan outline certain elements of a strategy, such as the directorate's vision, 
mission, and goals, they do not include several other desirable 
characteristics of a strategic plan, such as a discussion of how the 
directorate will (1) measure its progress in developing the national 
preparedness system, (2) address risk as it relates to preparedness 
activities, (3) coordinate with its preparedness stakeholders in developing 
and carrying out the various elements of the national preparedness 
system, and (4) integrate the elements of the national preparedness 
system. For example, FEMA has not included information on 
performance measures for meeting one of the objectives outlined in the 
operating plan — to support an integrated planning system for the federal 
preparedness-related agencies that links to regional, state, and local 
planning activities. The operating plan also does not define the problem or 
assess the risks that FEMA's national preparedness program faces. 
Specifically, it does not describe the threats, vulnerabilities, and 
consequences of a major homeland security incident or what FEMA's 
approach will be for addressing risk through its national preparedness 
system activities. While the draft operating plan identifies subcomponents 
in the Preparedness Directorate that will be responsible for carrying out 
segments of the national preparedness system, it does not discuss the 
roles and responsibilities of preparedness stakeholders, the coordination 
that will occur between them, or how the four elements of the national 
preparedness system will be integrated. 

The Post-Katrina Act calls on FEMA to develop and coordinate the 
implementation of a strategy for preparedness, and the complexity and 
difficulty of developing a national preparedness system underscore the 



In 2004, DHS drafted a strategy for a national preparedness system. The strategy, called 
Unified National Preparedness Strategy, aimed to unify and integrate planning, exercises, 
and assessments. However, the plan was not completed. Other programs in DHS that 
involve interagency coordination and cooperation at the federal level have developed 
strategic plans, for example, The National Strategy for Maritime Security and The 
National Strategy for the Physical Protection of Critical Infrastructures and Key Assets. 
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importance of strategic planning. 124 125 An important element of strategic 
planning is that it presents an integrated system of high-level decisions 
that are reached through a formal, visible process. The resulting strategy 
is thus an effective tool with which to communicate the mission and 
direction to preparedness stakeholders. The conditions we describe 
earlier in this report — such as incomplete plans on roles and 
responsibilities, unresolved corrective actions from exercises, and 
potential difficulties and historical delays in capability assessments — show 
that FEMA faces significant challenges in developing the key elements of 
the national preparedness system. 



The Desirable In 2004, we identified six desirable characteristics of an effective national 

Characteristics of a strategy that help achieve strategy success. These characteristics are 

National Strategy Can Help summarized in table 3. 126 

FEMA's National 

Preparedness Directorate 

in Developing a Strategic 

Plan 



124 6 U.S.C. § 313(b)(2)(H). 

125 The Department of Homeland Security's Office of Inspector General identified the need 
for a strategic plan in reviewing FEMA preparedness for catastrophic events in March 
2008. See DHS OIG, FEMA's Preparedness for the Next Catastrophic Disaster 
(Washington, D.C.: March 2008). 

126 Appendix IV provides more information on these six characteristics. 
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Table 3: Summary of Desirable Characteristics for a National Strategy 



Desirable characteristic Description 

Purpose, scope, and Addresses why the strategy was produced, the scope 

methodology of its coverage, and the process by which it was 

developed. 

Problem definition and risk Addresses the particular national problems and threats 
assessment the strategy is directed towards. 

Goals, subordinate objectives, Addresses what the strategy is trying to achieve, steps 
activities, and performance to achieve those results, as well as the priorities, 
measures milestones, and performance measures to gauge 

results. 

Resources, investments, and Addresses what the strategy will cost, the sources and 
risk management types of resources and investments needed, and 

where resources and investments should be targeted 
based on balancing risk reductions with costs. 

Organizational roles, Addresses who will be implementing the strategy, what 

responsibilities, and their roles will be compared to others, and 

coordination mechanisms for them to coordinate their efforts. 

Integration and implementation Addresses how a national strategy relates to other 

strategies' goals, objectives, and activities, and to 
subordinate levels of government and their plans to 
implement the strategy. 



Source: GAO data. 



We believe these characteristics can assist responsible parties, such as 
FEMA, in further developing and implementing national strategies as well 
as enhancing these strategies' usefulness for policy decisions to help 
achieve program results and accountability. The characteristics are a 
starting point for developing a strategic plan. However, we believe that an 
approach incorporating the substance of these characteristics is likely to 
increase success in strategy implementation. The following describes how 
each of these characteristics applies to the work of FEMA's National 
Preparedness Directorate. 

Purpose, scope, and methodology: National preparedness is an 
important part of homeland security efforts outlined in legislation, 
presidential directives, and policy documents. The National Strategy for 
Homeland Security recognized the importance of fostering a Culture of 
Preparedness that permeates all levels of our society, including all 
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preparedness stakeholders. 127 In summarizing lessons learned from 
Hurricane Katrina, the White House report made over 100 
recommendations and concluded that an immediate priority for correcting 
the shortfalls in the federal response to Hurricane Katrina was to define 
and implement a comprehensive national preparedness system. We 
believe a strategic plan for implementing the national preparedness system 
that includes a clearly stated purpose, scope, and methodology could help 
the Preparedness Directorate convey to preparedness stakeholders the 
importance of integrating the multiple elements of the national 
preparedness system and interagency coordination. 

Problem definition and risk assessment: As shown by 9/11 and 
Hurricane Katrina, the nation faces risks from terrorist attacks and man- 
made and natural disasters. These threats may vary between localities and 
regions, but responders are to be able to effectively work together in a 
common language on operational tasks when required. According to the 
National Preparedness Guidelines, responders must identify and assess 
risk to ensure the necessary capabilities are available for selecting the 
appropriate response. 128 In addition, understanding risk involves assessing 
what vulnerabilities and weaknesses require further attention. The lessons 
learned from Hurricane Katrina show that federal agencies were not 
prepared for a catastrophic disaster. Confusion by emergency responders 
over their roles and responsibilities was widespread and resulted in a slow 
or fragmented response. To improve response and recovery to all hazards 
including a catastrophic disaster, the Post-Katrina Act called on FEMA to 
develop the national preparedness system. 

Goals, subordinate objectives, activities, and performance 
measures: The cultural shift of the preparedness community from a 
response and recovery strategy to a proactive preparedness strategy 
emphasizes the importance of a strategic plan that includes clear goals, 
objectives, activities, and measures. Identifying performance measures for 
the various components of the national preparedness system, which is also 
a requirement under the Post-Katrina Act, will help policymakers 
determine what progress has been made and what remains to be done, 
especially as it relates to preparedness for a catastrophic disaster. 129 A 



Homeland Security Council, National Strategy for Homeland Security (Washington 
D.C.: October 2007). 

128 DHS, National Preparedness Guidelines (Washington, D.C.: September 2007). 
129 6 U.S.C. § 749(b). 
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strategic plan that outlines an overarching goal, subordinate objectives, 
activities, and performance measures for the various components of the 
national preparedness system would help FEMA prioritize future efforts 
and allow decision makers to measure progress. 

Resources, investments, and risk management: Preparedness 
agencies are to manage their likely risks and direct finite resources to the 
most urgent needs. The national preparedness system helps inform 
decision makers in federal and state agencies on their use of resources 
relative to their level of capabilities achievement. Different states and 
areas face different risks, and thus should have different capabilities to 
mitigate those risks. As we reported in March 2008, although DHS has 
taken some steps to establish goals, gather information, and measure 
progress, its monitoring of homeland security grant expenditures does not 
provide a means to measure the achievement of desired program 
outcomes. FEMA's current efforts do not provide information on the 
effectiveness of those funds in improving the nation's capabilities or 
reducing risk. 130 The National Strategy for Homeland Security describes 
how resources and risk management must be addressed in a 
comprehensive approach. 131 For example, the strategy states that "We 
must apply a risk-based framework across all homeland security efforts in 
order to identify and assess potential hazards, determine what levels of 
relative risk are acceptable, and prioritize and allocate resources among 
all homeland security partners, both public and private, to prevent, protect 
against, and respond to and recover from all manner of incidents." A 
strategic plan that outlines resources, investments, and risk management 
would help FEMA coordinate a prioritized approach. 

Organizational roles, responsibilities, and coordination: Achieving 
national preparedness, especially for catastrophic incidents, requires 
sharing responsibility horizontally with other federal departments and 
agencies. It also requires a robust vertical integration of the federal, state, 
local, and tribal governments, as well as private entities. FEMA's 
Preparedness Directorate faces the challenge of aligning operations of the 
nation's preparedness stakeholders to coordinate activities and plans to 



GAO, Homeland Security: DHS Improved its Risk-Based Grant Programs' Allocation 
and Management Methods, But Measuring Programs' Impact on National Capabilities 
Remains a Challenge, GAO-08-488T (Washington, D.C.: Mar. 11, 2008). 

131 Homeland Security Council, National Strategy for Homeland Security (Washington, 
D.C.: October 2007). 
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implement a national preparedness program capable of dealing with 
catastrophic incidents. A key part of a national preparedness strategic 
plan would be the clear delineation of organizations and their roles and 
responsibilities, as well as processes to coordinate their responsibilities. 

Integration and implementation: A national preparedness strategic 
plan would help describe how preparedness agencies at all government 
levels and sectors will integrate their various standards, policies, and 
procedures into the national preparedness system. Plans describing how 
to integrate and implement the various elements of the national 
preparedness system would help FEMA inform emergency managers, first 
responders, and decision makers on how the individual elements of the 
national preparedness system will improve capabilities, training, and plans 
for all hazards, including catastrophic disasters. A strategic plan to 
implement the national preparedness system would enable FEMA's 
Preparedness Directorate to improve its likelihood of achieving its vision, 
evaluating progress, and ensuring accountability of federal agencies and 
other organizations in aligning their efforts to develop and improve the 
national preparedness system. While it may be impossible to have 
absolute compatibility because of the many public and private 
organizations involved, the danger in organizations using different 
methods or systems, without some overall guidance to assure consistent 
application of approaches, is that the elements of the national 
preparedness system will have little ability to inform one another. More 
importantly, these systems may produce unreliable or incomplete data on 
how to improve programs related to response and recovery. FEMA plays a 
crucial role in this regard through its statutory responsibility of carrying 
out the Post-Katrina Act requirements for the national preparedness 
system. 



The nation looks to FEMA for leadership to ensure that stakeholders 
involved in preparedness activities can effectively provide a coordinated 
response to man-made or natural disasters. The nation's experiences after 
the events of 9/11 and the 2005 hurricane season dramatically 
demonstrated our emergency preparedness capabilities and where they 
were lacking. The Post-Katrina Act's centralization of responsibility in 
FEMA for exercises and the other primary activities that form the national 
preparedness system provides an unprecedented opportunity for 
comprehensive integration and coordination. While FEMA has made 
progress in implementing each of these interdependent and essential 
preparedness activities, it is difficult to measure this progress. FEMA 
lacks a comprehensive approach to managing the development of policies 



Page 69 



GAO-09-369 National Preparedness 



and plans and overseeing the National Exercise Program. Additionally, 
FEMA has not established a clearly defined course of action to assess 
capabilities based on quantifiable metrics. Finally, FEMA has not 
established a strategic plan for integrating these elements of the national 
preparedness system. These conditions show that much remains to be 
done. In the short term, progress is heavily dependent on continuing to 
improve basic policies and procedures, management tools, and project 
plans for key elements of the system. Developing each element of the 
system is undoubtedly a complex task, but progress has to be built on 
these incremental but critical steps. In the long term, progress will be 
increasingly dependent on how well FEMA coordinates with the 
thousands of stakeholders in the system and the degree to which it can 
integrate the plans, exercises, and assessments into a cohesive approach 
that improves national preparedness. This need centers attention on 
leadership and guidance from the Preparedness Directorate and success 
will depend on linking the various elements of the system and showing 
how data and information from the system will inform program and 
budget decisions related to improving preparedness. 

A complete, integrated set of national preparedness policies and plans that 
defines stakeholders' roles and responsibilities at all levels is needed to 
ensure that federal, state, and local resources are invested in the most 
effective exercises. As one program official told us, "If an exercise is 
testing an inadequate plan, then the exercise is just an experiment...." 
Until all national preparedness policies are developed and operational 
plans are created or revised to reflect changes in the roles and 
responsibilities of key stakeholders, FEMA's ability to update requisite 
training to prepare officials responsible for fulfilling these roles and reflect 
the preparedness lessons of the unprecedented disasters of the last decade 
in exercises and real-world response will be limited. Without a program 
management plan, FEMA cannot effectively ensure, in coordination with 
DHS and other federal entities, that it will complete and integrate key 
policies and plans with each other and the national preparedness system 
as envisioned by law and presidential directive. 

In implementing the latest iteration of the National Exercise Program, 
FEMA has issued guidance and requirements for exercise design, 
execution, evaluation, and corrective action resolution. However, federal 
and state exercise officials have not yet fully embraced the essential 
program components. In addition, opportunities to make exercises as 
realistic as possible by coordinating more fully with all preparedness 
stakeholders, including the National Council on Disability, and translating 
the experiences of real-world incidents into corrective actions could 
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further enhance the value of the exercise program. Establishing policies 
and procedures that detail how FEMA would work with federal entities as 
well as monitor states' compliance in implementing the program would 
help stakeholders meet program requirements and FEMA develop 
complete and accurate information on program implementation. More 
importantly, these key program controls, once more systematically 
established and applied, will enhance FEMA's ability to assess the extent 
to which corrective actions have been implemented and, ultimately, 
describe strengths and weaknesses in the nation's preparedness 
capabilities. 

Because a comprehensive assessment of national preparedness 
capabilities is a monumental task, it is understandable that FEMA's efforts 
to develop and implement an assessment approach have been underway 
for more than a decade. Program officials with responsibility for this most 
recent effort have recognized the need for a comprehensive set of metrics 
to identify needed capabilities in equipment, personnel, skills, or 
processes and to prioritize national investments in preparedness. Given 
the complexity of this effort, they would benefit from a clear roadmap that 
details their analytical approach for integrating disparate information 
sources, identifies associated program risks, establishes more specific 
milestones to help avoid unexpected setbacks, and provides a basis for 
assessing program progress and making revisions, if needed, to the 
agency's implementation plans. Such mitigation efforts would not 
eliminate the risks associated with the development of the comprehensive 
assessment system and target capabilities metrics, but they would provide 
a basis for holding officials responsible for timely and quality results and 
hold them harmless for unavoidable or unforeseen events that could delay 
their efforts. 

Finally, effective coordination, integration, and implementation of these 
elements of the national preparedness system require the combined 
contributions of a broad range of federal, state, and local stakeholders. 
FEMA has started this integration effort and has had some success in 
issuing guidelines and requirements that seek greater uniformity of effort. 
But our work shows that issuing guidelines alone does not assure 
consistent application across organizations. While it may be impossible to 
have absolute compatibility because of the many public and private 
organizations involved in preparedness, the danger in organizations using 
different methods or systems — without some overall guidance, direction, 
and controls in place to assure consistent application of preparedness 
approaches — is that the elements of the preparedness system will have 
little ability to inform one another. Perhaps, more important, these 
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systems may produce unreliable or incomplete data on how to allocate 
resources or to improve programs related to response and recovery, 
especially with respect to catastrophic incidents. In this regard, FEMA 
officials have noted that their authority is limited to coordinating with, but 
not directing, other federal agencies. This condition highlights the 
importance of developing a strategic approach that leads to partnerships 
with stakeholders whose cooperation is necessary for developing the 
preparedness system. Making progress with regard to this challenge is a 
necessary step to assessing our nation's capabilities and dealing with gaps 
in preparedness. The scope and breadth of this critical national effort 
suggests that an explicit description and elaboration of the elements of the 
system and the level of effort associated with its effective application 
could enhance stakeholder acceptance and participation. In addition, 
defining the end state of the preparedness system will help translate 
requirements from presidential directives and the Post-Katrina Act into 
measurable steps for achieving an integrated national system. Developing 
goals and metrics to measure progress towards achieving an integrated 
system will help FEMA prioritize actions, requirements, and national 
investments in preparedness. A strategic plan for the National 
Preparedness Directorate that describes how it will approach these 
challenges and mitigate these weaknesses would help FEMA partner with 
the many organizations whose cooperation and resources are necessary 
for success. 



Recommendations for ^° ensure * na * ^ e y e l emen ts of the national preparedness system are 

developed in a timely and integrated fashion, we recommend that the 

Executive Action Administrator of the Federal Emergency Management Agency take the 

following 11 actions: 



Direct the Disaster Operations Directorate and the National Preparedness 
Directorate to improve their approach to developing policies and plans 
that define roles and responsibilities and planning processes. 

• Develop a program management plan, in coordination with DHS and 
other federal entities, to ensure the completion of the key national 
preparedness policies and plans called for in legislation, presidential 
directives, and existing policy and doctrine, to define roles and 
responsibilities and planning processes, as well as to fully integrate 
such policies and plans into other elements of the national 
preparedness system. The program management plan, among other 
things, should: 
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• identify the specific schedule of activities that needs to be performed 
to complete, and identify dependencies among, all policy and planning 
development and integration activities; 

• identify the type and quantities of resources required to perform, and 
the schedule for completing, all policy and planning development and 
integration activities; 

• analyze activity sequences, durations (including the time required to 
partner and coordinate on an interagency basis with other federal 
entities), resource requirements, and schedule constraints to create 
and update the individual policy and plan development project 
schedules; and 

• control for changes to the project schedules precipitated by outside 
forces. 

When outstanding policies and plans are completed, integrate them into 
training and exercise efforts to ensure that roles and responsibilities are 
fully communicated and fully understood by emergency response 
stakeholders. 

Direct the National Exercise Division to improve its implementation of 
statutory and program requirements. 

• Coordinate with the Department of Homeland Security to develop 
policies and procedures for issuing after-action reports for National 
Level Exercises (i.e., TOPOFF) in 6 months or less, as required by the 
implementation plan for the National Exercise Program. 

• Collaborate with the Homeland Security Council to establish policies 
and procedures for documenting corrective actions from Principal 
Level Exercises that are consistent with HSEEP guidance and the 
implementation plan for the National Exercise Program. 

• Collaborate with the Homeland Security Council to provide FEMA with 
the information it needs from past principal level exercises to enable it 
to conduct remedial action tracking and long-term trend analysis, as 
required by the Post-Katrina Act. 

• Ensure compliance by states that receive grant funds with HSEEP 
requirements by revising FEMA's grant monitoring guidance, for 
example by including a checklist of specific HSEEP requirements for 
state validation and certification. 

• Involve the National Council on Disability on committees involved in 
the design and execution of national level exercises, especially on 
issues related to populations with special needs. 

• Develop internal control policies and procedures that validate the 
completeness and accuracy of data used to measure program 
performance. Such procedures could involve checking whether states 
and federal agencies are providing data and information needed to 
measure the performance of the program. 
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• Revise the National Exercise Program Implementation Plan to require 
the use of FEMA's Corrective Action Program for all federal exercises 
that involve interagency testing of roles and responsibilities or require 
that federal agencies submit a report to FEMA on the status of their 
corrective actions resulting from such exercises. 

• Develop procedures for including "lessons learned" from real-world 
incidents in the Corrective Action Program system. 

Direct the Office of Preparedness Policy, Planning, and Analysis to 
improve its approach for developing a comprehensive assessment system. 

• Enhance its project management plan to include milestone dates, an 
assessment of risk, and related mitigation strategies for (1) 
comprehensively collecting and reporting on disparate information 
sources, (2) developing quantifiable metrics for target capabilities that 
are to be used to collect and report preparedness information, and (3) 
reporting on the results of preparedness assessments to help inform 
homeland security resource allocation decisions. 

Direct the National Preparedness Directorate to take a more strategic 
approach to developing the national preparedness system. 



• Develop a strategic plan for implementing the national preparedness 
system that includes the key characteristics of a strategic plan, 
including coordination, integration, and implementation approaches. 



Agency Comments 
and Our Evaluation 



We provided a draft of this report to DHS for comment. In commenting on 
our draft report, DHS noted that while it may not agree with all the 
assertions in the report, it generally agreed with our recommendations. 
DHS stated that GAO's recommendations provide a useful methodology 
and sound counsel for revision of FEMA's current portfolio of national 
preparedness policy, plans, protocols, and procedures. Specifically, DHS 
stated that FEMA has already made significant inroads in each aspect of 
the GAO recommended characteristics for sound strategic planning. 



DHS also expressed concern that the report suggests that DHS/FEMA 
should hold other federal agencies and departments or state, local, or 
tribal governments accountable for compliance with program 
requirements, while also recognizing that FEMA did not generally have the 
explicit authority to compel compliance. The Post-Katrina Act designates 
FEMA as the federal leader and coordinator for developing and 
implementing the national preparedness system. We recognize that 
FEMA's authority is generally to coordinate, guide, and support, rather 
than direct, and that collaboration is an essential element of FEMA's 



Page 74 



GAO-09-369 National Preparedness 



efforts. At the same time, we believe that FEMA's expanded leadership 
role under the Post-Katrina Act provides FEMA opportunities for and a 
responsibility to further develop its relationships with national 
preparedness stakeholders at the local, state, and federal levels and to 
instill a shared sense of responsibility and accountability on the part of all 
stakeholders for the successful development and implementation of the 
national preparedness system. Several of our recommendations aim to 
enhance such collaboration and cooperation. 

DHS also provided technical comments, which we incorporated into the 
report as appropriate. Appendix V contains written comments from DHS. 

As agreed with your office, unless you publicly announce its contents 
earlier, we plan no further distribution of this report until 30 days from the 
date of this report. At that time we will send copies of this report to the 
Secretary of Homeland Security, the Director of the Office of Management 
and Budget, and interested congressional committees. We will also make 
copies available to others on request. In addition, this report will be 
available at no charge on the GAO Web site at http://www.gao.gov. 

If you or your staff have any questions concerning this report, please 
contact me at (202) 512-8757 or e-mail atjenkinswo@gao.gov. Contact 
points for our Offices of Congressional Relations and Public Affairs may 
be found on the last page of this report. Key contributors to this report are 
listed in appendix VI. 




William 0. Jenkins, Jr. 

Director, Homeland Security and Justice Issues 
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Appendix I: Objectives, Scope, and 
Methodology 



This review examined key elements of the national preparedness system, 
including the National Exercise Program. Specifically, our reporting 
objectives were to review the extent to which the Federal Emergency 
Management Agency (FEMA) has: 

1. developed policies and plans that define roles and responsibilities and 
planning processes for national preparedness; 

2. taken actions since 2007 to implement the National Exercise Program 
and track corrective actions at the federal and state levels and what 
challenges remain; 

3. made progress in conducting a nationwide capabilities-based 
assessment, including developing required preparedness reports, and 
what issues, if any, it faces in completing the system; and, 

4. developed a strategic plan for implementing the national preparedness 
system. 

To address these objectives, we analyzed information and data on FEMA's 
policies and plans for preparedness, the National Exercise Program, its 
approach for developing a comprehensive system for assessing nationwide 
capabilities, and its strategy for integrating elements of the preparedness 
system. GAO explored the option of selecting exercises to review, but 
there is no national database that captures all exercises conducted using 
Homeland Security Grant Program funds. Therefore we selected six 
states — California, Georgia, Illinois, New York, Texas, and Washington — 
that provide examples of how exercises are planned and conducted and 
visited these six states. While we cannot generalize our work from these 
visits to all states, we chose these locations to provide examples of the 
way in which states carry out their exercise and preparedness programs. 
In selecting these states, we considered factors such as states' 
participation in national-level exercises; states located in different 
geographic locations, such as those in hurricane-prone regions; and states 
with varying percentages of homeland security grant funding planned to 
support exercises. At each location, we interviewed staff in FEMA's 
regional offices responsible for regional preparedness activities. We 
interviewed state and local officials on their progress and challenges in 
carrying out preparedness activities, including exercises and assessments 
of capability. We analyzed key legislation such as the Post-Katrina Act and 
the Implementing Recommendations of the 9/11 Commission Act of 2007 1 
(9/11 Act) as well as presidential directives related to preparedness efforts. 



'Pub. L. No. 110-53, 121 Stat. 266 (2007). 
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Methodology 



We also interviewed FEMA officials responsible for preparedness 
programs to learn more about the actions they had taken and planned to 
take related to preparedness efforts and compared FEMA's policies and 
procedures with criteria in GAO's standards for internal control in the 
federal government. 2 

To analyze the extent to which policies and plans have been developed to 
define roles and responsibilities and planning processes for national 
preparedness, we analyzed key legislation, presidential directives, and 
DHS- and FEMA-issued policies that identify required preparedness 
policies and plans to define roles and responsibilities for emergency 
response as well as establish guidance for planning processes for 
developing emergency response plans. We identified the resulting policies 
that define roles and responsibilities and that form the basis of the 
national preparedness system, including the National Strategy for 
Homeland Security, the National Response Framework (NRF), the 
National Preparedness Guidelines, and the National Incident Management 
System. In addition, we identified related policies that supplement these 
documents, such as guidance for Joint Field Office operations. We also 
identified policies that define planning processes for developing 
emergency plans, such as the draft Integrated Planning System (IPS) and 
FEMA-issued interim and final Comprehensive Preparedness Guides for 
nonfederal planning efforts. We identified plans developed using planning 
processes that further define and operationalize roles and responsibilities 
identified in existing policies. These plans include the incident annexes 
and the incident annex supplements for the NRF, FEMA's Pre-Scripted 
Mission Assignment Catalog, plans being developed as part of FEMA's 
Catastrophic Disaster Planning Initiative, as well as plans called for by 
HSPD 8 Annex 1 that are to be developed using IPS. To identify lessons 
learned and corrective actions related to roles and responsibilities from 
federal emergency response exercises, we summarized lessons learned 
from after-action reports for Tier I and II (or equivalent) exercises from 
2005 through 2008. The exercises that comprised this data set were 



2 GAO Standards for Internal Control in the Federal Government, GAO/AIMD-00-21.3.1 
(Washington, D.C.: November 1999). These standards, issued pursuant to the requirements 
of the Federal Managers' Financial Integrity Act of 1982 (FMFIA), provide the overall 
framework for establishing and maintaining internal control in the federal government. 
Also pursuant to FMFIA, the Office of Management and Budget (OMB) issued Circular A- 
123, revised December 21, 2004, to provide the specific requirements for assessing the 
reporting on internal controls. Internal control standards and the definition of internal 
control in OMB Circular A-123 are based on GAO's Standards for Internal Control in the 
Federal Government. 
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identified by FEMA officials as well as counsel to the White House 
Homeland Security Council. This analysis was conducted to determine 
whether the exercises revealed unclear or conflicting roles and 
responsibilities between federal departments and agencies and if 
additional policies and plans were needed. We also interviewed officials 
from DHS's Office of Operations Coordination and FEMA's National 
Preparedness Directorate and Disaster Operations Directorate to obtain 
information on the status of efforts to develop and issue required 
preparedness policies and plans, including any existing program or project 
management plans and related issuance schedules. We compared policies 
and plans that have been published in a final form, versus released in 
interim or draft formats or that have not yet been developed, to determine 
the issuance status (completed, partially completed, or incomplete) of 
these policies and plans. To identify best practices for program 
management, such as steps for how a program is to be executed, 
monitored, and controlled, we reviewed the Project Management 
Institute's The Standard on Program Management. Finally, we reviewed 
our prior reports on FEMA's preparedness programs and planning efforts, 
as well as prior DHS, White House, and congressional reports on the 
lessons learned from the response to Hurricane Katrina in 2005. 

To assess the extent to which FEMA has taken actions since 2007 to 
implement a National Exercise Program and track corrective actions, we 
observed portions of two National Exercise Program exercises (TOPOFF 4 
and the May 2008 Tier II exercise). Specifically, during TOPOFF 4, we 
discussed exercise implementation with federal, state, and local officials 
and observed FEMA's exercise management efforts at the TOPOFF 4 
Master Control Cell in Springfield, Virginia; the TOPOFF 4 Long-Term 
Recovery Tabletop Exercise in Washington, D.C.; and exercise 
implementation in Portland, Oregon. During the May 2008 Tier II exercise, 
we observed exercise implementation in Mount Weather and Suffolk, 
Virginia and in Blaine, Washington and discussed the exercise with 
participating federal, state, and local officials. We evaluated key program 
documents, such as the implementation plan for the National Exercise 
Program and the Homeland Security Exercise and Evaluation Program 
(HSEEP) — FEMA's guidance for carrying out exercises in accordance with 
the Post-Katrina Act and the 9/11 Act and data on the program's 
performance measures. We reviewed actions taken by FEMA since 2007 
because the National Exercise Program Charter was established in 
January 2007 and the implementation plan was issued in April 2007. We 
examined after-action reports for Principal Level Exercises— exercises 
which involve senior federal officials, such as Deputy Secretaries of 
departments or agencies— that were issued from April 2007 through August 
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2008 to determine whether the Homeland Security Council developed 
after-action reports. 3 We also interviewed Homeland Security Council 
staff and the Associate Counsel to the President on the role and 
responsibility of the council for systematically tracking and implementing 
corrective actions resulting from Principal Level Exercises because these 
staff were responsible for summarizing corrective actions for Principal 
Level Exercises. We also reviewed after-action reports that were provided 
to us by the six states we visited for exercises conducted from June 2007 
through September 2008 that used Homeland Security Grant Program 
funds, in order to determine how well these states were complying with 
HSEEP and grant guidance. To determine if FEMA is conducting 
monitoring and oversight of Homeland Security Grant Program recipients, 
we reviewed grant monitoring reports for the six states we visited. For 
information that would provide a broader perspective on FEMA's efforts, 
we examined several FEMA databases, including the FEMA Secure 
Portal — the FEMA repository of after-action reports; the National Exercise 
Schedule (NEXS) system — a scheduling system for all exercises; and the 
Corrective Action Program (CAP) system — which is designed for tracking 
capability-based improvement plans entered by federal, state, and local 
exercise participants. We assessed the reliability of the FEMA Secure 
Portal and NEXS databases by checking the systems to determine if 
known exercises identified through after-action reports produced by 
states were included in these systems, and by interviewing FEMA and 
state officials responsible for the data. In addition, we assessed the 
reliability of the CAP system by interviewing FEMA and Homeland 
Security Council officials responsible for the data. We concluded the data 
in the FEMA Secure Portal, the NEXS system, and the CAP system were 
not reliable for use in this report because these databases lacked complete 
information related to after-action reports, scheduled exercises, and 
corrective actions and FEMA does not have procedures in place to ensure 
that required data are collected consistently to populate these databases. 

To determine the extent to which FEMA has made progress and issues 
FEMA has encountered in conducting a nationwide capabilities-based 
assessment and developing required preparedness reports and any issues 



3 For the purposes of this report, we reviewed exercises conducted after the approval of the 
April 2007 National Exercise Program Implementation Plan and conducted before 
September 2008 because the implementation plan requires after action-reports for Tier I 
level exercises, including Principle Level Exercises, be issued within 180 days, or 6 months, 
after the completion of an exercise. After-action reports for exercises occurring after 
August 2008 were not required to be completed until after the end of our audit work. 
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it faces in completing the system, we (1) analyzed FEMA's plans and 
schedules for developing the comprehensive assessment system and 
performance objectives for measuring capabilities, including assessment 
efforts initiated by FEMA's National Preparedness, Disaster Operations, 
and Grant Programs Directorates, and (2) interviewed FEMA staff 
responsible for these efforts. We also reviewed assessments previously 
conducted by DHS and FEMA to evaluate historical efforts to assess 
capabilities. To assess FEMA's efforts to establish quantifiable metrics for 
target capabilities, we analyzed preliminary performance objectives for 
two target capabilities that FEMA had completed, and interviewed 
headquarters staff responsible for these efforts. We also reviewed 
information pertinent to FEMA's assessment approach, including the 
Federal Preparedness Report (issued by FEMA in January 2009) and State 
Preparedness Reports for the 2007 reporting year for the six states we 
visited (reports for 2008 were due to FEMA after this report was finalized 
for publication). To assess the comparability of information contained in 
the six State Preparedness Reports, we selected one target capability — 
mass prophylaxis — outlined in the Target Capabilities List, and reviewed 
a performance measure associated with this capability for providing initial 
prophylaxis (an action taken to prevent a disease or a health problem) 
within 48 hours of a state/local decision to provide prophylaxis. To 
identify best practices for project management, such as steps for how a 
project is to be executed, monitored, and controlled, we reviewed the 
Project Management Institute's A Guide to the Project Management Body 
of Knowledge (PMBOK) and compared FEMA's efforts for developing and 
implementing the comprehensive assessment system to the best practices 
developed by the institute. Finally, we reviewed our prior GAO reports on 
FEMA's preparedness programs and exercise efforts. 

To determine the extent to which FEMA's National Preparedness 
Directorate has developed a strategic plan that implements the national 
preparedness system, we interviewed FEMA National Preparedness 
Directorate officials on strategic planning and policy and procedures for 
the National Preparedness System. We analyzed the Post-Katrina Act and 
key FEMA documents — including the agencywide Strategic Plan for Fiscal 
Years 2008 through 2013, the Grants Program Directorate Strategic Plan, 
and the draft annual National Preparedness Directorate Operating Plan — 
to determine strategy-related requirements. Additionally we reviewed 
DHS and DHS Office of Inspector General reports on national 
preparedness. To determine the elements that comprise a strategic plan, 
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we examined our prior reports on the desirable characteristics of effective 
national strategies and compared them with FEMA's current approach for 
developing a National Preparedness System. 4 Finally, we compared the 
desirable characteristics to the work of the National Preparedness 
Directorate as described in the Homeland Security Council's National 
Strategy for Homeland Security, DHS's National Preparedness Guidelines, 
and the Post-Katrina Act. 

We conducted this performance audit from January 2008 through April 
2009 in accordance with generally accepted government auditing 
standards. Those standards require that we plan and perform the audit to 
obtain sufficient, appropriate evidence to provide a reasonable basis for 
our findings and conclusions based on our audit objectives. We believe 
that the evidence obtained provides a reasonable basis for our findings 
and conclusions based on our audit objectives. 



4 GAO, Combating Terrorism: Evaluation of Selected Characteristics in National 
Strategies Related to Terrorism, GAO-04-408T (Washington, D.C.: Feb. 3, 2004). 



Page 81 



GAO-09-369 National Preparedness 



Appendix II: Overview of Policies and Plans to 
Define Roles and Responsibilities and Planning 
Processes for Developing Emergency Plans 



In September 2006, we reported that in preparing for, responding to, and 
recovering from any disaster, the legal authorities, roles and 
responsibilities, and lines of authority for the preparation and response at 
all levels for government and nongovernmental entities must be clearly 
defined, effectively communicated, and well understood in order to 
facilitate rapid and effective decision making. 1 National preparedness 
policies and plans identify these legal authorities, roles and 
responsibilities, and lines of authority for response activities and serve to 
communicate this information to emergency response stakeholders, and, 
in conjunction with training, are the basis for ensuring that the 
information is well understood. Effective and efficient disaster 
management relies on the thorough integration of these emergency 
response policies and plans. Because emergency response activities entail 
large numbers of stakeholders who need to be able to respond to an 
incident in a coordinated and integrated manner, it is essential that roles 
and responsibilities are defined, communicated, and understood prior to a 
real-world incident response. An example of the range of stakeholders 
involved in such response activities is illustrated by figure 6. 



^AO, Catastrophic Disasters: Enhanced Leadership, Capabilities, and Accountability 
Controls Will Improve the Effectiveness of the Nation's Preparedness, Response, and 
Recovery System, GAO-06-618 (Washington, D.C.: Sept. 6, 2006). 
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Figure 6: Selected Stakeholders That Have Roles and Responsibilities for Emergency Response Activities 




Selected Members of the Emergency Response Community 



Source: GAO analysis. 
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Note: NGO = Non-Governmental Organization. 

Figure 6 presents a range of emergency response stakeholders, but does not attempt to depict the 
operational or information-sharing relationships or relative authorities of such stakeholders. 



The wide range of emergency response stakeholders depicted in figure 6, 
among others, are to be organized by the roles and responsibilities defined 
in policies and plans that are designed to facilitate an effective response to 
an incident, be it man-made (e.g., terrorism) or a natural disaster. Policies 
that broadly define roles and responsibilities are operationalized by the 
development of plans that provide greater levels of detail. 2 These detailed 
plans are to be developed using the planning processes that are discussed 
and established in federal policies. The range and relative relationships of 
the policies that define roles and responsibilities and planning processes 
for developing emergency plans are illustrated in figure 7. Table 4, which 
follows figure 7, describes each of the policies presented in the graphic. 



2 We use the term "policy" in this report to include certain laws, executive orders, and 
derivative policy documents that are relevant to national preparedness, including 
emergency response. 
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Figure 7: Range and Relative Relationship of Policies That Define Roles and Responsibilities and Planning Processes for 
Developing Emergency Plans 
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Among the 50 policies that define roles and responsibilities or planning 
processes, 46 define roles and responsibilities and 4 define planning 
processes for developing emergency plans. Table 4 provides brief 
descriptions and the status of the policies depicted in figure 7 that define 
roles and responsibilities. Of the 46 policies presented in table 4, 40 have 
been completed and 6 are incomplete. 



Table 4: Description and Status of the 46 Policies That Define Roles and Responsibilities 



Type 8 Title 



Summary 



Status" 



Policy & doctrine (Legislation) 



Robert T. Stafford Disaster 
Relief and Emergency 
Assistance Act 

(Stafford Act), 

Pub. L. No. 93-288 



The Stafford Act, as amended, primarily establishes the programs and 
processes for the federal government to provide major disaster and 
emergency assistance to states, local governments, tribal nations, 
individuals, and qualified nonprofit organizations. Among other things, 
the act defines the role of the Federal Coordinating Officer (FCO), who 
is to coordinate major disaster and emergency assistance upon 
appointment by the President, as well as the role of the FEMA 
Administrator, who is responsible for emergency preparedness and is 
vested with the authority to prepare federal response plans. 



Completed 
(enacted May 1 974) 



Homeland Security Act of The Homeland Security Act established the Department of Homeland Completed 
2002, Security (DHS), with the Secretary of Homeland Security as the head (enacted November 

Pub L No 107-296 of the department, through the merger of disparate agencies and 2002) 

organizations, including FEMA. It generally charged DHS with securing 
the homeland against terrorist attacks and carrying out the functions of 
all transferred entities, including acting as a focal point regarding 
natural and man-made crises and emergency planning. 



Post-Katrina Emergency 
Management Reform Act 
of 2006 

(Post-Katrina Act), 
Pub. L. No. 109-295, 
Title VI 



The Post-Katrina Act, among other things, enhanced FEMA's 
responsibilities and its autonomy within DHS. Under the act, FEMA's 
primary mission is to lead and support the nation in a risk-based, 
comprehensive emergency management system of preparedness, 
protection, response, recovery, and mitigation. It directs the transfer to 
FEMA of many functions of DHS' former Preparedness Directorate, 
and contains a provision establishing in FEMA a National Integration 
Center, which is responsible for the ongoing management and 
maintenance of the National Incident Management System (NIMS) and 
the National Response Plan (NRP), which is now known as the 
National Response Framework (NRF). 



Completed 

(enacted October 
2006) 
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Type" Title 

Implementing 
Recommendations of the 
9/1 1 Commission Act of 
2007 

(9/1 1 Act), 

Pub. L No. 110-53 



Summary 

The 9/1 1 Act was enacted to implement recommendations in a variety 
of areas that were made by the National Commission on Terrorist 
Attacks Upon the United States. Among other things, the act amends 
certain provisions of the Homeland Security Act and the Post-Katrina 
Act in order to strengthen the use of the incident command system. 
For example, the act requires FEMA Regional Administrators to assist 
state, local, and tribal governments to preidentify and evaluate suitable 
sites where a multijurisdictional incident command system may be 
quickly established and operated, if the need arises. 



Status" 

Completed 

(enacted August 
2007) 



Policy and doctrine (Presidential Directives) 



Homeland Security 
Presidential Directive 5 

(HSPD 5) 

Management of Domestic 
Incidents 



Titled "Management of Domestic Incidents," the purpose of this HSPD 
is to enhance the ability of the United States to manage domestic 
incidents by establishing a single, comprehensive national incident 
management system (NIMS) and national response plan (NRP), both 
of which are required by the Homeland Security Act. HSPD-5 also 
defines the Secretary of Homeland Security as the principal federal 
official for domestic incident management and as responsible for 
coordinating federal operations within the United States to prepare for, 
respond to, and recover from terrorist attacks, major disasters, and 
other emergencies, pursuant to the Homeland Security Act. 



Completed 
(February 2003) 



Homeland Security 
Presidential Directive 8 

(HSPD 8) 

National Preparedness 



Titled "National Preparedness," this HSPD establishes policies to 
strengthen the preparedness of the United States by requiring a 
national domestic all-hazards preparedness goal, establishing 
mechanisms for improved delivery of federal preparedness assistance 
to state and local governments, and outlining actions to strengthen 
preparedness capabilities of federal, state, and local entities. It 
requires the head of each federal department or agency to undertake 
actions to support the national preparedness goal, including adoption 
of quantifiable performance measurements in the areas of training, 
planning, equipment, and exercises for federal incident management 
and asset preparedness. 



Completed 
(December 2003) 
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Type" Title 

Homeland Security 
Presidential Directive 8 
Annex 1 

(HSPD 8 Annex 1) 
National Planning 



Summary 

Titled "National Planning," this HSPD is intended to further enhance 
the preparedness of the United States by formally establishing, 
developing, and maintaining a standard and comprehensive approach 
to national planning. It calls for: 

• a standardized federal planning process (the Integrated 
Planning System); 

• national planning doctrine; 

• resourced operational and tactical planning capabilities at 
each federal department and agency with a role in homeland 
security; 

• strategic guidance statements, strategic plans, concepts of 
operations, operations plans, and, as appropriate, tactical 
plans for each National Planning Scenario; and 

• a system for integrating plans among all levels of 
government. 

It also calls for the development of the National Homeland Security 
Plan. 



Status" 

Completed 
(December 2007) 



Policy and doctrine (Issued by DHS or FEMA) 



National Strategy for 
Homeland Security 
(NSHS) 



The NSHS is to guide, organize, and unify homeland security efforts 
by providing a common framework for the prevention of terrorist 
attacks; protection of people, critical infrastructure, and key resources; 
and response to and recovery from man-made and natural disasters. 
It calls for homeland security management through a continuous, 
mutually reinforcing cycle of four activity phases: 

1 . overarching homeland security guidance grounded in clearly 
articulated and up-to-date homeland and relevant national 
security policies, with coordinated supporting strategies, 
doctrine, and planning guidance flowing from and fully 
synchronizing with these policies; 

2. a deliberate and dynamic system that translates policies, 
strategies, doctrine, and planning guidance into a family of 
strategic, operational, and tactical plans; 

3. the execution of operational and tactical-level plans; and 

4. continual assessment and evaluation of both operations and 
exercises. 



Completed 

(July 2002; revised 
October 2007) 
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Type" Title 

National Homeland 
Security Plan (NHSP) 



Summary 

NHSP, as called for under HSPD 8 Annex 1 , is to be an overarching 
strategic plan to guide national efforts to execute the National Strategy 
for Homeland Security. It is intended to: 

• facilitate federal homeland security coordination, 

• establish priorities, and 

• define roles and responsibilities for preventing, protecting 
against, responding to, and recovering from man-made and 
natural disasters. 

It was to be submitted to the President for approval within 120 days of 
the approval of HSPD 8 Annex I (December 2007); however, as of 
April 2009, the NHSP has yet to be published. 



Status" 

Incomplete: 

not yet published 

(release schedule 
undetermined) 



National Incident NIMS presents a core set of doctrine, concepts, principles, Completed 

Management System procedures, organizational processes, terminology, and standard (March 2004' revised 

(NIMS) requirements designed to enable effective, efficient, and collaborative December 2008) 

incident management. It forms the basis for interoperability and 
compatibility to enable a diverse set of public and private organizations 
to conduct integrated emergency management and incident response 
operations. 



National Preparedness 
Guidelines (NPG) 

and 3 capabilities-based 
preparedness tools 

• Universal Task List, 

• Target Capabilities 
List, and 

• National Planning 
Scenarios 



The NPG consists of a vision, capabilities, and priorities for national 
preparedness. It establishes three capabilities-based preparedness 
tools (Universal Task List, Target Capabilities List, and National 
Planning Scenarios) and the National Preparedness System cycle 
(plan; organize and staff; equip; train; and exercise, evaluate, and 
improve) to collate existing homeland security plans, strategies, and 
systems into an overarching framework. The revised and finalized 
NPG replaces the interim National Preparedness Goal released in 
March 2005. 



Completed 
(interim NPG, March 
2005; final NPG and 3 
capabilities-based 
preparedness tools, 
September 2007) 



National Response 
Framework (NRF) 

base document 



The NRF is a guide to how the nation conducts all-hazards response, 
generally describing national response doctrine and the roles and 
responsibilities of officials involved in response efforts, including, 
among others, the Secretary of Homeland Security, FEMA 
Administrator, Principal Federal Official, and Federal Coordinating 
Officer. It is designed to align key roles and responsibilities, linking all 
levels of government, nongovernmental organizations, and the private 
sector, as well as capture specific authorities and best practices for 
managing incidents that range from the serious but purely local, to 
large-scale terrorist attacks or catastrophic natural disasters. 



Completed 
(January 2008) 
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Type" Title 

15 NRF Emergency 
Support Function (ESF) 
Annexes 



Summary 

NRF ESF Annexes align categories of federal government response 
resources and capabilities and provide strategic objectives for their 
use under the NRF. They provide the structure for coordinating federal 
interagency support for a federal response to an incident and are 
mechanisms for grouping functions most frequently used to provide 
federal support to states, and federal-to-federal support. Each ESF 
Annex, such as Search and Rescue, identifies the federal agency 
coordinator and the primary and support agencies pertinent for the 
ESF, and when activated, the initial actions delineated in the ESF 
Annexes guide response activities. 



Status" 

Completed 
(January 2008) 



8 NRF Support Annexes NRF Support Annexes describe the roles and responsibilities of Completed 

federal departments and agencies, nongovernmental organizations, (January 2008) 

and the private sector in coordinating and executing the functional 

processes and administrative requirements necessary for incident 

management that are common to all incidents. They identify the 

federal agency coordinator and the primary and support agencies 

pertinent for the support activity, such as financial management or 

private-sector coordination, and when activated, the initial actions 

delineated in the Support Annexes guide response activities. 



4 NRF Partner Guides 



NRF Partner Guides are to provide stakeholder-specific references 
describing key roles and actions for local, tribal, state, federal, private- 
sector, and nongovernmental response partners. They are to 
summarize core NRF concepts and be tailored specifically to leaders 
at different levels of government and from different types of 
organizations. As of April 2009, none of the four NRF Partner Guides 
have been published. 



Incomplete: 

none published 

(release schedule 
undetermined) 



Joint Field Office 
Interagency Integrated 
Standard Operating 
Procedure (JFO IISOP) 

JFO Appendixes & 
Annexes (JFO A&A) 

JFO Field Operations 
Guide (JFO FOG) 



The various Joint Field Office (JFO) guides were written to support the 
December 2004 National Response Plan for the establishment of 
JFOs. The JFO is a temporary federal multi-agency coordination 
center established locally to facilitate coordinated field-level domestic 
incident management activities, with the JFO IISOP, JFO A&A, and 
JFO FOG designed to assist personnel assigned to response 
operations. In particular, the JFO IISOP and JFO A&A provide 
detailed guidance on JFO activations and operations, including 
defining the roles and responsibilities and concept of operations for the 
Principal Federal Official cell within the JFO. The JFO FOG is 
intended to be used as a quick reference job aid for JFO personnel. 



Completed 
(April 2006) 



Completed 
(April 2006) 

Completed 
(June 2006) 



JFO Organization and 
Functions Manual 



According to FEMA officials, FEMA's Disaster Operations Directorate 
is developing a JFO Organization & Functions Manual, reflecting the 
NRF, as an addition to existing JFO guidance. This new manual is to 
provide guidance for the establishment, operation, and demobilization, 
as well as the general organization and staffing, of JFOs 



Incomplete: 

not yet published 

(release schedule 
undetermined) 



Source: GAO analysis of legislation, presidential directives, and policy issued by DHS and FEMA. 
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" The type column identifies the categorization of policy and plans. Policies are categorized under the 
term "policy & doctrine" (including legislation, presidential directives, and other policies that define 
roles and responsibilities, as well as policies that define planning processes), per the national 
preparedness cycle presented in the National Preparedness Guidelines. Plans that define roles and 
responsibilities and that are to operationalize policies are categorized under the term "planning & 
resource allocation," also per the national preparedness cycle presented in the National 
Preparedness Guidelines. 

b The status column identifies the categorization of policy and plans as "completed," "partially 
completed," or "incomplete." Completed policies and plans are those that have been publicly issued 
in a final form. Partially completed policies and plans are those that have been issued in interim 
formats or fully drafted and revised but not yet published. Incomplete policies and plans are those 
that have not been issued in an interim or draft format, or issued in a final form. 

c When the status of a policy or plan identifies that the release schedule is undetermined, this means 
that DHS, FEMA, or other responsible parties for the development of a particular policy or plan have 
not determined the final release date for the publication. 

Table 5 provides brief descriptions and the status of the policies that 
define planning processes for developing the emergency plans depicted in 
figure 7. Of the four policies presented in the table, two have been 
completed and two are partially completed. 
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Table 5: Description and Status of the Four Policies That Define the Planning Processes to Be Used for Developing 
Emergency Plans 



Type" Title 
Policy and doctrine 



Summary 



Status" 



Operational Planning 
Guidance for FEMA 
Operational Planners 



According to FEMA officials, this document provides FEMA operational 
planners with guidance and priorities for their operational planning efforts. This 
guidance applies to operational planners in both FEMA headquarters and 
FEMA regional offices (including the Operational Planning Branch of FEMA's 
Disaster Operations Directorate and for FEMA's Catastrophic Disaster 
Planning Initiative). According to FEMA officials, this guidance was issued in 
March 2009 and will be updated at the beginning of each fiscal year. 



Completed: 
(March 2009) 



Integrated Planning 
System (IPS) 



IPS, as called for in HSPD 8 Annex I, is to provide common processes for 
developing emergency plans and is to include: 

• national planning doctrine and planning guidance, instruction, and 
processes to ensure consistent planning across the federal 
government; 

• a mechanism that provides for concept development to identify and 
analyze missions and potential courses of action; 

• a description of the process that allows for plan refinement and proper 
execution to reflect developments in risk, capabilities, or policies, as 
well as to incorporate lessons learned from exercises and actual 
events; 

• a description of the process that links regional, state, local, and tribal 
plans, planning cycles, and processes and allows these plans to 
inform the development of federal plans; 

• a process for fostering integration of federal, state, local, and tribal 
plans that allows for state, local, and tribal capability assessments to 
feed into federal plans; and 

• a guide for all-hazards planning, with comprehensive, practical 
guidance and instruction on fundamental planning principles that can 
be used at federal, state, local, and tribal levels to assist the planning 
process. 

It was to be submitted to the President for approval within 2 months of the 
approval of HSPD 8 Annex I, which was signed by the President in December 
2007. According to FEMA officials, IPS received presidential approval in 
January 2009 under President Bush; however, IPS has not been publicly 
released as of April 2009. The Obama administration is currently conducting a 
review of the document. 



Partially completed: 

draft approved by 
President Bush in 
January 2009 

(currently being 
reviewed by 
President Obama's 
administration, 
release schedule 
undetermined) 



Comprehensive 
Preparedness Guide 
101 (CPG 101) 

Developing and 
Maintaining 
Emergency Plans 



The CPG 101 provides state and local communities with guidance for 
emergency operations planning and describes how the state and local planning 
process is to vertically integrate with the federal Integrated Planning System. It 
is designed to promote a common understanding of the fundamentals of 
emergency planning to help emergency planners examine a hazard and 
produce integrated, coordinated, and synchronized plans. The CPG includes 
planning processes for all mission areas — prevention, protection, response, 
and recovery. 



Completed: 

(interim CPG 101 
July 2008; final CPG 
101 April 2009) 
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Type" Title 



Summary 



Status" 



Interim 

Comprehensive 
Preparedness Guide 
301 (CPG 301) 

Planning Guide for 
Special Needs 
Populations 



The interim CPG 301 is the second guide being released by FEMA to provide 
comprehensive guidance on developing emergency plans (interim CPG 101 
being the first). It is intended as a tool for state, territorial, tribal, and local 
emergency managers in the development of emergency operations plans that 
address planning for special needs populations. The guide outlines how 
involving special needs populations in planning considerations enables 
emergency managers to address the function-based needs of individuals and 
offers scalable recommendations to meet the needs of different jurisdictions. 
This document supplements the prior FEMA guidance document 
"Accommodating Individuals with Disabilities within Disaster Mass Care, 
Housing, and Human Services." 



Partially completed: 
interim version 
released August 
2008 

(final release 

schedule 

undetermined) 



Source: GAO analysis of legislation, presidential directives, and policy issued by DHS and FEMA. 

° The type column identifies the categorization of policy and plans. Policies are categorized under the 
term "policy & doctrine" (including legislation, presidential directives, and other policies that define 
roles and responsibilities, as well as policies that define planning processes), per the national 
preparedness cycle presented in the National Preparedness Guidelines. Plans that define roles and 
responsibilities and that are to operationalize policies are categorized under the term "planning & 
resource allocation," also per the national preparedness cycle presented in the National 
Preparedness Guidelines. 

b The status column identifies the categorization of policy and plans as "completed," "partially 
completed," or "incomplete." Completed policies and plans are those that have been publicly issued 
in a final form. Partially completed policies and plans are those that have been issued in interim 
formats or fully drafted and approved but not yet published (e.g., the Integrated Planning System). 
Incomplete policies and plans are those that have not been issued in an interim or draft format, or 
issued in a final form. 

c When the status of a policy or plan identifies that the release schedule is undetermined, this means 
that DHS, FEMA, or other responsible parties for the development of a particular policy or plan have 
not determined the final release date for the publication. 



The range and relative relationships of the plans that define roles and 
responsibilities are illustrated in figure 8. Table 6, which follows figure 8, 
describes each of the plans presented in the graphic. 
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Figure 8: Range and Relative Relationship of Plans That Define Roles and Responsibilities 
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State and Local Emergency 
Operations Plans 



Plans 

J Plans that define roles and responsibilities 



Source: GAO analysis. 



Table 6 provides brief descriptions and the status of the plans that define 
roles and responsibilities that are depicted in figure 8. Of the 72 plans 
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presented in the table, 20 have been completed, 3 have been partially 
completed, and 49 are incomplete. 



Table 6: Description and Status of the 72 Plans That Define Roles and Responsibilities 

Type 8 Title Summary Status" 

Planning and resource allocation 

7 NRF Incident Annexes Seven NRF Incident Annexes describe the policies, concept of 

operations, and responsibilities of emergency response stakeholders 
to address specific contingency or hazard situations or an element of 
an incident requiring specialized application of the NRF. As of April 
2009, two of seven incident annexes have not been published — the 
Terrorism Incident Law Enforcement and Investigation Annex and 
the Cyber Incident Annex. Until revised, the December 2004 
National Response Plan versions of these documents remain in 
effect. 



Completed: 5 of 7 
incident annexes 

Incomplete: 2 of 7 
incident annexes 

(release schedule 
undetermined) 



2 NRF Incident Annex 
Supplements: 

• Catastrophic Incident 
Supplement (CIS) 

• Mass Evacuation 
Incident Annex 
Operational 
Supplement 
(MEIAOS) 



The supplement (CIS) to the NRF Catastrophic Incident Annex has 
yet to be fully revised and published under the NRF. The CIS was 
originally published under the NRP in September 2006 and describes 
the coordinated strategy for accelerating the delivery and application 
of federal and federally accessible resources and capabilities in 
response to a catastrophic event. According to FEMA officials, the 
operational annexes to the CIS (Execution Schedule and 
Transportation Support Schedule) are currently being updated to 
reflect the present response capabilities of the federal government. 
In addition, the future revision of the base CIS document under the 
NRF will reflect FEMA's current broadening of the scope and 
application of existing response mechanisms to be utilized for 
catastrophic disaster response. 

The MEIAOS has yet to be published under the NRF. According to 
FEMA officials, this supplement to the NRF Mass Evacuation 
Incident Annex has been drafted and is currently undergoing internal 
review at FEMA. The MEISAOS is intended to provide additional 
guidance for mass evacuations of large numbers of people in 
incidents requiring a coordinated federal response. 



Incomplete: 

CIS not yet fully 
revised and published 
under the NRF 

(release schedule 
undetermined) 

Incomplete: 

MEIAOS not yet 
published 

(release schedule 
undetermined) 



FEMA Pre-Scripted 
Mission Assignment 
(PSMA) Catalog 



When the President declares a major disaster or emergency under 
the Stafford Act, FEMA has the authority to direct other federal 
entities to provide assistance to affected jurisdictions through the 
issuance of mission assignments (MA). In order to expedite the 
delivery of federal assistance, and as required by the Post-Katrina 
Act, FEMA developed the PSMA Catalog. PSMAs provide standard 
statements of work and cost estimates that can be used to develop 
MAs for other federal entities, and the use of PSMA language 
streamlines the MA process to enable quicker federal response 
times. PSMAs also provide a planning base for federal agencies. 
FEMA published the PSMA Catalog in December 2008, with a total 
of 236 pre-scripted mission assignments across 29 federal 
departments and agencies. 



Completed 
(December 2008) 
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6 FEMA Regional 
Hurricane Contingency 
Plans (RHCPS) 



According to FEMA officials, six FEMA Regions (I, II, III, IV, VI, and 
IX) have developed RHCPS in coordination with their regional 
nonfederal response partners. These plans are designed to 
establish a coordinated approach for the delivery and application of 
federal resources and capabilities to support response to tropical 
storms and hurricanes affecting the FEMA Regions. RHCPS identify 
critical actions to be coordinated at the regional level. The actions, 
priorities, and timelines identified in the RHCPS are designed to 
provide guidance to all stakeholders on readiness, response, and 
initial recovery actions. 



Completed: 

6 of 6 RHCPS 
finalized and 
published 



Florida Catastrophic 
Comprehensive Response 
Plan - 

FEMA Catastrophic 
Disaster Planning Initiative 



This plan is being developed through the use of scenario-based and Incomplete: 



required-resource planning processes applied to the state of Florida 
to enhance its capability to respond to catastrophic events. The 
planning initiative will result in a set of plans, including county, 
regional, state, and supporting federal response plans for a 
catastrophic event impacting South Florida, including a category five 
hurricane. The final version of the plan is scheduled to be released 
in July 2009. 



not yet published 

(final plan scheduled 
for release in July 
2009) 



Draft Hawaii Hurricane 
Contingency Plan - 

FEMA Catastrophic 
Disaster Planning Initiative 



According to FEMA officials, a draft plan has been developed for a 
category four/five hurricane strike on the southern part of the 
Hawaiian island of Oahu. A major focus of the planning effort 
involves the challenge of providing rapid large-scale disaster relief to 
the geographically isolated island. The schedule for release of the 
final version of the plan is undetermined. 



Partially completed: 
draft plan published 
(final plan release 
schedule 
undetermined) 



Interim Federal 
Contingency Plan (FCP) - 
New Madrid Seismic Zone 
Catastrophic Earthquake 



The interim FCP addresses major issues the federal government 
expects to encounter for a no-notice catastrophic earthquake in the 
New Madrid Seismic Zone (NMSZ), including direction and control 
for response operations for the eight states and four FEMA regions in 
the NMSZ. Major components of the plan include action checklists, 
timelines, senior leadership issues, and operational tools and 
information for senior management. The final FCP, which is currently 
under development, is intended to allow emergency managers to 
develop detailed response plans and for FEMA and other federal 
entities to develop supporting response plans to meet state capability 
shortfalls. The I FCP will be replaced by the integrated concept of 
operations and operational final FCP, scheduled for release by May 
2010 — one year prior to the national level exercise scheduled for 
201 1 that will test the plan. 



Partially completed: 

interim FCP published 
June 2008 

(final FCP scheduled 
for release by May 
2010) 
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2 California (CA) 
Earthquake Contingency 
Plans - 

FEMA Catastrophic 
Disaster Planning Initiative 



A California Catastrophic Disaster Planning Initiative is currently 
underway. This two-phase initiative involves scoping activities to 
determine catastrophic seismic incident readiness for response 
planning in both Northern and Southern California. Phase one of the 
initiative resulted in the creation of a San Francisco Bay Area 
Earthquake Contingency Plan. Phase two of the initiative is currently 
underway, focusing on the development of a contingency plan for 
responding to a magnitude 7.0 earthquake along the San Andreas 
Fault in southern CA. The schedule for the release of this plan is 
September 2010. 



Completed: 
Northern CA plan 
published 2008 

Incomplete: 
Southern CA plan not 
published 

(scheduled for release 
by September 2010) 



Draft Northwest Nevada 
Earthquake Contingency 
Plan - 

FEMA Catastrophic 
Disaster Planning Initiative 



Prompted by a series of magnitude 2.5 to 4.1 earthquakes in 
northwest Nevada, this planning initiative resulted in the creation of a 
Draft Northwest Nevada Earthquake Contingency Plan. This plan 
was tested during the 2008 Vigilant Guard exercise, which was 
designed to test the capabilities and interoperability of first 
responders at the local, regional, and state levels in Nevada in 
conjunction with the Nevada National Guard. 



Partially completed: 

draft published 2008 

(final release 

schedule 

undetermined) 



8 National Planning 
Scenario (NPS) Strategic 
Guidance Statements 
(SGS) 



According to HSPD 8 Annex I, after the approval of the Integrated 
Planning System, the Secretary of Homeland Security is to develop a 
strategic guidance statement (SGS) for each of the National Planning 
Scenarios (NPS). The eight consolidated NPS are intended to focus 
national planning efforts on the most likely or most dangerous threats 
to the homeland. The SGS are to be documents that outline 
strategic priorities, broad national strategic objectives, and basic 
assumptions; describe the envisioned end state; and establish the 
general means necessary to accomplish that end for each NPS. 
HSPD 8 Annex I does not establish a completion date expectation for 
the development of the SGS for each NPS. According to DHS 
officials, as of April 2009, four SGS have been approved and one is 
under development (drafted and undergoing interagency review and 
adjudication). 



Completed: 
4 SGS 

Incomplete: 

4 of 8 SGS not yet 
published 

(see figure 9 below for 
release schedule) 



8 NPS Strategic Plans 
(STRATPLANS) 



According to HSPD 8 Annex I, no later than 90 days after the 
approval of each strategic guidance statement (SGS) for each 
National Planning Scenario (NPS), the Secretary of Homeland 
Security is to develop corresponding strategic plans (STRATPLANS) 
for each SGS. The STRATPLANS are to be plans that define the 
mission, identify authorities, delineate roles and responsibilities, 
establish mission-essential tasks, determine required and priority 
capabilities, and develop performance and effectiveness measures 
for each NPS. The STRATPLANS are effective as of their approval 
by the Secretary for Homeland Security. According to DHS officials, 
as of April 2009 two STRATPLANS have been approved and two are 
under development (drafted and undergoing interagency review and 
adjudication). 



Completed: 

2 STRATPLANS 

Incomplete: 

6 of 8 STRATPLANS 
not yet published 

(see figure 9 below for 
release schedule) 
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8 NPS Concept Plans 
(CONPLANS) 



According to HSPD 8 Annex I, no later than 180 days after the 
approval of each strategic plan (STRATPLAN) for each National 
Planning Scenario (NPS), the Secretary of Homeland Security is to 
develop corresponding concept plans (CONPLANS) for each 
STRATPLAN. The CONPLANS are to be plans that briefly describe 
the concept of operations for integrating and synchronizing existing 
federal capabilities to accomplish the mission-essential tasks, and 
describe how federal capabilities will be integrated into and support 
regional, state, local, and tribal plans for each NPS. The CONPLANS 
are effective as of their approval by the Secretary for Homeland 
Security. According to DHS and FEMA officials, as of April 2009 one 
CONPLAN has been approved and two are under development 
(drafted and are undergoing interagency review and adjudication). 



Completed: 
1 CONPLAN 

Incomplete: 

7 of 8 CONPLANS 
not yet published 

(see figure 9 below for 
release schedule) 



16 NPS Operational Plans 
(OPLANS) & Tactical 
Plans" 



According to HSPD 8 Annex I, no later than 120 days after the 
approval of each concept plan (CONPLAN) for each National 
Planning Scenario (NPS), the head of each federal agency with a 
role in homeland security is to develop corresponding operations 
plans (OPLANS) and, at his or her discretion, a tactical plan to 
execute the roles and responsibilities assigned to that agency in 
each CONPLAN for each NPS. The OPLANS are to be plans that 
identify detailed resource, personnel, and asset allocations in order 
to execute the objectives of the strategic plan and turn strategic 
priorities into operational execution, and contain the full description of 
the concept of operations, to include specific roles and 
responsibilities, tasks, integration, and actions required, with 
supplemental support function annexes as appropriate. The tactical 
plans contain the detailed development and identification of 
individual tasks, actions, and objectives tailored to specific situations 
and fact patterns at an operational level, and are to support and 
achieve the objectives of the OPLANS. According to DHS officials, 
as of April 2009, no OPLANS or tactical plans have been approved 
under HSPD 8 Annex I. 



Incomplete: 

none of the 16 
OPLANS published 

(see figure 9 below for 
release schedule) 



10 FEMA Regional All- 
Hazards Response Plans 



According to FEMA officials, each FEMA Region is responsible for 
developing a regional all-hazards response plan that details the 
specific regional-level actions and activities taken by federal entities 
to support state and territorial requirements for emergency response. 
These plans are to include organizational coordination mechanisms 
and implementing instructions for accomplishing the actions agreed 
upon for joint federal and state operations during disasters, and are 
to provide the necessary link between state emergency operations 
plans and federal plans, in accordance with the NRF. 



Incomplete: 

none of the 10 FEMA 
Regional All-Hazards 
Response Plans have 
been published 

(release scheduled for 
June 2009) 



Source: GAO analysis of legislation, presidential directives, and policy issued by DHS and FEMA. 

" The type column identifies the categorization of policy and plans. Policies are categorized under the 
term "policy & doctrine" (including legislation, presidential directives, and other policies that define 
roles and responsibilities, as well as policies that define planning processes), per the national 
preparedness cycle presented in the National Preparedness Guidelines. Plans that define roles and 
responsibilities and that are to operationalize policies are categorized under the term "planning & 
resource allocation," also per the national preparedness cycle presented in the National 
Preparedness Guidelines. 
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" The status column identifies the categorization of policy and plans as "completed," "partially 
completed," or "incomplete." Completed policies and plans are those that have been publicly issued 
in a final form. Partially completed policies and plans are those that have been issued in interim 
formats or fully drafted and revised but not yet published (e.g., Draft Northwest Nevada Earthquake 
Contingency Plan). Incomplete policies and plans are those that have not been issued in an interim 
or draft format, or issued in a final form. 

c When the status of a policy or plan identifies that the release schedule is undetermined, this means 
that DHS, FEMA, or other responsible parties for the development of a particular policy or plan have 
not determined the final release date for the publication. 

d As of April 2009, the total number of operational plans to be developed per HSPD 8 Annex 1 is 
uncertain. We determined that 1 6 operational plans need to be developed as required by Annex 1 . 
We calculated this number by counting DHS and FEMA as entities that both need to develop 
operational plans to carry out their roles and responsibilities for each of the eight consolidated 
national planning scenarios (NPS) under the planning process established by Annex 1 . In addition to 
DHS and FEMA, it is likely that other federal departments and agencies will also have to develop 
operational plans to address their roles and responsibilities for response to each of the eight 
consolidated NPS, as appropriate. For example, other federal entities, such as DOD, EPA, and the 
Coast Guard responded to Hurricane Katrina and would likely need to develop an operational plan 
per Annex 1 for response to a major hurricane. However, while the exact number of operational 
plans that federal departments and agencies will need to develop as required by Annex 1 is unknown, 
it is likely to be higher than the 1 6 we used in our calculation. 

As noted in table 6, a schedule exists for the release of the myriad plans 
called for under HSPD 8 Annex 1 using the not yet published Integrated 
Planning System (IPS). Figure 9 presents the schedule for the release of 
specific plans under IPS. 
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Figure 9: Schedule for the Development and Finalization of Plans Called for Under HSPD 8 Annex 1, Utilizing the Integrated 
Planning System 



HSPD 8 Annex 1 Plan 


Strategic Guidance 
Statement (SGS) 
Status 


Strategic Plan 
(STRATPLAN) 
Status 


Concept Plan 
(CONPLAN) 
Status 


Operational Plan 
(OPLAN) 
Status 


Entity Responsible for 
Plan Development 


DHS Operations 

\s\J\Ji Ui / 1 all %JI 1 al fU 

Planning based 
Interagency Incident 
Management Planning 
Team 


DHS Operations 

Ul/l//uniaiiUil CZffU 

Planning based 
Interagency Incident 
Management Planning 
Team 


FEMA Disaster 
Operations Directorate 


Federal departments 
and agencies 


National Planning 
Scenario 


Terrorist Use of 
Explosives Attack 


Approved by Secretary of 
Homeland Security — 
August 2008 


Approved by Secretary of 
Homeland Security — 
November 2008 


Approved by Secretary of 
Homeland Security — 
April 2009 


Under development — 
started January 2009 


Improvised Nuclear 
Device Attack 


Approved by Secretary of 
Homeland Security — 
beptemDer ^uuo 


Approved by Secretary of 
Homeland Security — 
January ^uuy 


Under development: 
undergoing interagency 
review/adjudication — 
as of March 2009 


Awaiting development — 
projected start date 
undetermined 


Biological Attack 


Approved by Secretary of 
Homeland Security — 
January 2009 


Under development: 
undergoing interagency 
review/adjudication — 
as of March 2009 


Under development: 
undergoing interagency 
review/adjudication — 
as of March 2009 


Awaiting development — 
projected start by 
April 2009 


Radiological Dispersion 
Device Attack 


Approved by Secretary of 
Homeland Security — 
January 2009 


Under development: 
undergoing interagency 
review/adjudication — 
as of March 2009 


Awaiting development — 
projected start by 
April 2009 


Awaiting development — 
projected start by 
July 2009 


Chemical Attack 


Under development: 
undergoing interagency 
review/adjudication — 
as of March 2009 


Awaiting development — 
projected start by 
April 2009 


Awaiting development — 
projected start by 
August 2009 


Awaiting development — 
projected start by 
November 2009 


Natural Disasters - 
hurricanes and 
earthquakes 


Awaiting development — 
projected start by 
April 2009 


Awaiting development — 
projected start by 
May 2009 


Awaiting development — 
projected start by 
November 2009 


Awaiting development — 
projected start by 
February 2010 


Cyber Attack 


Awaiting development — 
projected start by 
July 2009 


Awaiting development — 
projected start by 
September 2009 


Awaiting development — 
projected start by 
February 2010 


Awaiting development — 
projected start by 
May 2010 


Pandemic Influenza 


Awaiting development — 
projected start by 
August 2009 


Awaiting development — 
projected start by 
January 2010 


Awaiting development — 
projected start by 
May 2010 


Awaiting development — 
projected start by 
August 2010 



Source: DHS and FEMA. 
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Finally, figure 10 shows the combined universe of all the policies and 
plans in figures 7 and 8 in relation to each other. Figure 11, which follows, 
shows the status of development of each of the policies and plans. 
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Figure 10: Range and Relative Relationship of Policies and Plans That Define Roles and Responsibilities and Planning 
Processes for Developing Emergency Plans 



LATION 
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National Homeland Security Plan 




National Response Framework base document 

- 15 NRF Emergency Support Function Annexes 
- 8 NRF Support Annexes 



- 7 NRF Incident Annexes 
■ 2 NRF Incident Annex Supplements 



- 4 NRF Partner Guides 



National Preparedness Guidelines 

- Universal Task List 
- Target Capabilities List 
- National Planning Scenarios 



Joint Field Office (JFO) 

Interagency 
Integrated Standard 
Operating Procedure 
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- JFO Field Operation 

Guide 
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Functions Manual 



Operational Planning Guidance for FEMA Operational Planners 



FEMA Pre-Scripted Mission 
Assignment Catalog 











6 FEMA 
Regional Hurricane 
Contingency Plans 


4- 




interim Federal 
Contingency Plan: 




-> 


New Madrid Seismic Zone 
Catastrophic Earthquake 










Florida Catastrophic 
Comprehensive 
Response Plan 






2 California Earthquake 


<- 


-> 


Contingency Plans 










draft Hawaii 
Hurricane 
Contingency Plan 
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Contingency Plan 


<- 


-> 



Integrated Planning System 



3 NPS Strategic Guidance Statements 

- 8 NPS Strategic Plans 

- 8 NPS Concept Plans 

16 NPS Operational STactical Plans 





r 


10 FEMA Regional 
All-Hazards Response Plans 
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r 


- Comprehensiv 
Guid 

- interm Comprehe 
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e Preparedness 
3 101 

nsive Preparedness 
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State and Local Emergency 
Operations Plans 



Policies and plans 

| Policies that define roles and responsibilities and planning processes 

I Plans 



Source: GAO analysis. 
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Figure 11: Status of Development of Policies and Plans That Define Roles and Responsibilities and Planning Processes for 
Developing Emergency Plans 



LEGISLATION 


Stafford Act 




Homeland Security Act 




Post-Katrina Act 




9/1 1 Act 


LEGISLATION 



Presidential 
Directives, 
DHS/FEMA 
Policy & 
Plans 



HSPD 5 

Management of Domestic Incidents 



HSPD 8 
National Preparedness 



HSPD 8 Annex 1 
National Planning 



National Strategy for Homeland Security 



National Homeland Security Plan 



> 


* 






National Incident 
Management System 




National Preparedness Guidelines 

- Universal Task List 
- Target Capabilities List 




National Response Framework base document 

- 15 NRF Emergency Support Function Annexes 
- 8 NRF Support Annexes 






- National Planning Scenarios 




- 7 NRF Incident Annexes 



- 2 NRF Incident Annex Supplements 
- 4 NRF Partner Guides 



Joint Field Office (JFO) 

Interagency 
Integrated Standard 
Operating Procedure 

- JFO Appendixes & 

Annexes 

- JFO Field Operation 

Guide 



- JFO Organization & 
Functions Manual 



T 



J 



FEMA Pre-Scripted Mission 
Assignment Catalog 



Operational Planning Guidance for FEMA Operational Planners 



6 FEMA 
Regional Hurricane 
Contingency Plans 



Florida Catastrophic 
Comprehensive 
Response Plan 



draft Hawaii 
Hurricane 
Contingency Plan 



interim Federal 
Contingency Plan: 
New Madrid Seismic Zone 
Catastrophic Earthquake 



2 California Earthquake 
Contingency Plans 



draft Northwest 
Nevada Earthquake 
Contingency Plan 



Integrated Planning System 

I NPS Strategic Guidance Statements 

- 8 NPS Strategic Plans 

- 8 NPS Concept Plans 



erational &' 



10 FEMA Regional 
ll-Hazards Response Plai 



Comprehensive Preparedness 
Guide 101 



Interm Comprehensive Preparedness 
Guide 301 



State and Local Emergency 
Operations Plans 



Status of Policies and Plan 3 

J Published documents 

| Draft or interim version completed, or partially completed for multiple deliverables 
I Draft or interim version not completed, not yet published 



Source: GAO analysis. 
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Note: Figure 1 1 provides a graphic depiction of the status of policies and plans that define roles and 
responsibilities and planning processes for developing emergency plans. 

• The policies and plans in figure 1 1 that are identified as « published » (white boxes with black 
text) align with the policies and plans that are identified as « completed » in tables 4, 5, and 6 of 
this appendix. 

• The policies and plans in figure 1 1 that are identified as « not yet completely drafted » or « not 
yet published » (green boxes with white text) align with the policies and plans that are identified 
as « incomplete » in tables 4, 5 and 6. 

• However, the policies and plans in figure 1 1 that are identified as « draft or interim version 
completed » or « partially completed for multiple deliverables » (purple boxes with white text) do 
not numerically align with the number of policies and plans that are identified as « partially 
completed » in tables 4, 5, and 6. Tables 4, 5, and 6 refer to policies and plans as « partially 
completed if the individual policy or plan has been issued in an interim format or fully drafted 
and revised but not yet published (e.g., draft Northwest Nevada Earthquake Contingency Plan). 
For figure 1 1 , the policies and plans identified as "partially completed" are grouped together with 
plans that have been completed for some but not all deliverables. For example, in figure 1 1 the 
7 NRF Incident Annexes are coded as partially completed for multiple deliverables (purple boxes 
with white text) because not all 7 NRF Incident Annexes are completed. Each box that is coded 
purple that contains multiple plans that are "partially completed for multiple deliverables" are 
detailed as follows in table 6: 

o Of the 7 NRF Incident Annexes, 5 are completed and 2 are incomplete. 

o Of the 2 California Earthquake Contingency Plans, 1 is completed and 1 is incomplete. 

o Of the range of plans called for under HSPD 8 Annex 1 : 

■ 8 NPS Strategic Guidance Statements: 4 are completed and 4 are 
incomplete. 

■ 8 NPS Strategic Plans: 2 are completed and 6 are incomplete. 

■ 8 NPS Concept Plans: 1 is completed and 7 are incomplete. 
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This appendix presents additional information regarding the Federal 
Emergency Management Agency's (FEMA) progress and any remaining 
issues it faces in conducting a nationwide comprehensive assessment 
system. FEMA has identified various ongoing and historical assessment 
efforts that it plans to use to inform the development of the 
comprehensive assessment system. Additional information regarding 
these efforts is outlined below. 

• State Preparedness Reports. While state preparedness reports 
assess capabilities within states, FEMA could not use information in 
the 2007 reports to compare capability gaps between states, because 
states did not report information using common metrics to assess 
capabilities and data were not always available to consistently 
complete the report. FEMA's state preparedness guidance explains 
that states are to "use relevant metrics . . . from the Target Capabilities 
List when describing current capabilities." However, FEMA has not 
developed a framework for states to use in reporting their current 
capabilities against the target capabilities because FEMA is in the 
process of (1) developing quantifiable metrics for the target capabilities 
and (2) revising the reporting format for state preparedness reports in 
order to base them on the target capabilities. As a result, the 2007 
reports do not report state capabilities in a measurable way, or with the 
level of detail necessary for a comparison across states and territories. 1 
In addition, the six states we visited used different techniques to 
summarize their capabilities. In one location, a state homeland 
security task force held discussion groups to determine what the 
capability needs are and what resources are needed. In another 
location, the state held a workshop attended by stakeholders from 
across the state to collect and obtain input on capability needs to 
complete the state preparedness report. Two states relied on 
information collected for their respective state homeland security 
strategies. A fifth state primarily used information it had gathered to 



x We reviewed the 2007 state preparedness reports for the six states we visited and selected 
one activity associated with the mass prophylaxis capability — percentage of at-risk 
population that was successfully provided with initial prophylaxis (an action taken to 
prevent a disease or a health problem) within 48 hours of state/local decision to provide 
prophylaxis. The Target Capabilities List indicates that 100 percent (referred to as a 
metric) of at-risk population is to be provided with initial prophylaxis within 48 hours of 
states and local governments' decision to do so (referred to as a performance measure). 
We found that capabilities information was not comparable across the six state reports. 
Three of the six state preparedness reports we reviewed described efforts to provide mass 
prophylaxis within 48 hours, while the other three state preparedness reports did not 
reference or describe their capability to carry out this activity. 
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prepare a grant funding reporting requirement, while officials at a sixth 
state collected information through site visits. FEMA headquarters 
officials explained that they intend to use the target capabilities as the 
framework for future state preparedness reports. 

• National Incident Management System Compliance Assessment 
Support Tool (NIMSCAST). NIMSCAST is a Web-based tool to 
assess states' and territories' compliance with the NIMS, which is a 
standardized process by which emergency responders are to conduct 
integrated incident response operations, rather than a method for 
assessing capabilities. Assessing compliance with National Incident 
Management System requirements is one of the requirements for the 
comprehensive assessment system. 2 In February 2009, FEMA 
indicated that it will use NIMSCAST to continue collecting data on 
NIMS compliance in addition to collecting capability and state 
preparedness report data through a survey it will distribute in 2009. 
FEMA noted that this effort will consolidate reporting requirements 
and fulfill Post-Katrina Act requirements for the comprehensive 
assessment system. In addition, FEMA noted that it will use data 
collected through NIMSCAST related to compliance with incident 
management processes and procedures to directly inform the analysis 
of two target capabilities: On-Site Incident Management and 
Emergency Operations Center Management. However, it is unclear 
how FEMA will integrate this tool with other features of the 
comprehensive assessment system to assess the remaining 35 
capabilities that will not be directly informed by NIMSCAST data 
related to compliance with incident management processes and 
procedures. 

• Gap Analysis Program (GAP). The methodology for the GAP 
analysis focuses on seven activities needed to respond to a hurricane 
or other disaster, such as sheltering and debris removal, but does not 
include all activities needed to address all 37 target capabilities. GAP 
was originally designed to provide a snapshot of gaps in disaster- 
response resources in hurricane-prone regions. In the first phase of the 
program, FEMA's Disaster Operations Directorate assessed the 
readiness of five hurricane-prone regions composed of 17 states, the 
District of Columbia, Puerto Rico, and the U.S. Virgin Islands in 2007. 
The Disaster Operations Directorate expanded the effort nationwide in 
2009 to address all hazards so that it can better assess readiness. 

• Cost-to-Capability Initiative (C2C). In 2008, FEMA's Grant 
Programs Directorate launched its C2C, the purpose of which is to help 



2 6 U.S.C. § 749(c)(1). 
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FEMA and localities better target and measure the results of using 
federal grant funds. As its organizing structure, the C2C uses the 15 
National Planning Scenarios, such as a hurricane, earthquake, 
improvised explosive device, or anthrax attack. Data for the C2C are to 
be based on self assessments of capabilities from state preparedness 
plans, estimates of baseline capability, and the estimated relative 
capability improvement expected from a requested level of grant 
investment. To be used effectively and enable comparisons across 
jurisdictions in evaluating grant proposals, the state and local data for 
assessing state and local capabilities must be in the common language 
of target capabilities and have metrics that are compatible with C2C. 
These metrics are being developed by FEMA's National Preparedness 
Directorate. However, grantee use of C2C will not be mandatory, and 
thus its ultimate value is yet to be determined. In developing the 
initiative, grant officials are considering ways to collect data from 
stakeholders with a minimal burden and integrate analyses resulting 
from the C2C into existing programs, plans, and procedures. 

• National Preparedness System (NPS). The NPS was discontinued 
because it was time consuming and did not produce meaningful data. 
This Web-based management information system was designed to 
serve as an inventory tool to measure a jurisdiction's ability to deliver 
elements of planning, organization, equipment, training, and exercises. 3 
Although it was developed in response to Homeland Security 
Presidential Directive 8's preparedness requirements, in conjunction 
with the Target Capabilities List, and pilot tested in 10 states, 4 the 
system was discontinued by the Department of Homeland Security 
because officials said it was too time consuming to use, according to 
FEMA officials. Because it was only piloted, FEMA officials explained 
that it did not generate meaningful preparedness information from the 
data collected. According to FEMA budget documentation required by 
the Office of Management and Budget for major information 
technology investments, FEMA spent nearly $15 million in total on the 
system for 2006, 2007, and 2008 before it was discontinued. 

• Pilot Capability Assessment (PCA). The PCA was labor intensive 
and did not generate meaningful data. This assessment, based on the 
37 target capabilities, was also intended to measure jurisdictions' 



The system is a stand-alone Web-based system, and is different from the National 
Preparedness System that is required by the Post-Katrina Act to ensure that the nation has 
the ability to deal with all-hazards incidents. 

4 The National Preparedness System was piloted in Alabama, California, Colorado, Idaho, 
Iowa, Maryland, New Hampshire, New Jersey, Rhode Island, and Virginia. 
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progress in achieving needed target capabilities. While it was 
developed in response to HSPD 8 preparedness requirements and in 
conjunction with the Target Capabilities List and pilot tested in six 
states, 5 FEMA officials said it was too labor intensive. Because it was 
only piloted, FEMA did not generate meaningful preparedness 
information from the data collected, according to FEMA officials. 
• Capability Assessment for Readiness (CAR). The CAR lacked 
controls for validating the accuracy of self-reported assessment data. 
This assessment was proposed as a one-time nationwide assessment of 
performance in areas such as planning and hazard management to 
assess a national set of emergency management performance criteria 
for FEMA grant recipients. FEMA committed to preparing this 
assessment in hearings before the Senate Committee on 
Appropriations. The assessment was conducted in 1997 but concerns 
reported by the DHS Inspector General in March 2006 regarding self- 
reporting and the lack of controls for validating information reported 
by states limited the reliability and, therefore, the value of the data. 
The assessment was conducted once in 1997. 



5 The Pilot Capability Assessment was pilot tested in Colorado, Florida, Michigan, 
Minnesota, Pennsylvania, and Utah. 
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In February 2004, we identified six desirable characteristics of an effective 
national strategy that would enable its implementers to effectively shape 
policies, programs, priorities, resource allocations, and standards and that 
would enable federal departments and other stakeholders to achieve the 
identified results. 1 We further determined in that report that national 
strategies with the six characteristics can provide policy makers and 
implementing agencies with a planning tool that can help ensure 
accountability and more effective results. To develop these six desirable 
characteristics of an effective national strategy, we reviewed several 
sources of information. First, we gathered statutory requirements 
pertaining to national strategies, as well as legislative and executive 
branch guidance. We also consulted the Government Performance and 
Results Act of 1993, general literature on strategic planning and 
performance, and guidance from the Office of Management and Budget on 
the President's Management Agenda. In addition, among other things, we 
studied past reports and testimonies for findings and recommendations 
pertaining to the desirable elements of a national strategy. Furthermore, 
we consulted widely within GAO to obtain updated information on 
strategic planning, integration across and between the government and its 
partners, implementation, and other related subjects. 

We developed these six desirable characteristics based on their underlying 
support in legislative or executive guidance and the frequency with which 
they were cited in other sources. We then grouped similar items together 
in a logical sequence, from conception to implementation. The following 
sections provide more detail on the six desirable characteristics. 

Purpose, scope, and methodology: This characteristic addresses why 
the strategy was produced, the scope of its coverage, and the process by 
which it was developed. For example, a strategy should discuss the 
specific impetus that led to its being written (or updated), such as 
statutory requirements, executive mandates, or other events like the global 
war on terrorism. Furthermore, a strategy would enhance clarity by 
including definitions of key, relevant terms. In addition to describing what 
it is meant to do and the major functions, mission areas, or activities it 
covers, a national strategy would ideally address its methodology. For 
example, a strategy should discuss the principles or theories that guided 



^AO, Combating Terrorism: Evaluation of Selected Characteristics in National 
Strategies Related to Terrorism, GAO-04-408T (Washington, D.C.: Feb. 3, 2004). 
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its development, the organizations or offices that drafted the document, or 
working groups that were consulted in its development. 

Problem definition and risk assessment: This characteristic addresses 
the particular national problems and threats at which the strategy is 
directed. Specifically, this means a detailed discussion or definition of the 
problems the strategy intends to address, their causes, and operating 
environment. In addition, this characteristic entails a risk assessment, 
including an analysis of the threats to and vulnerabilities of critical assets 
and operations. If the details of these analyses are classified or 
preliminary, an unclassified version of the strategy should at least include 
a broad description of the analyses and stress the importance of risk 
assessment to implementing parties. A discussion of the quality of data 
available regarding this characteristic, such as known constraints or 
deficiencies, would also be useful. 

Goals, subordinate objectives, activities, and performance 
measures: This characteristic addresses what the national strategy strives 
to achieve and the steps needed to garner those results, as well as the 
priorities, milestones, and performance measures to gauge results. At the 
highest level, this could be a description of an ideal end state, followed by 
a logical hierarchy of major goals, subordinate objectives, and specific 
activities to achieve results. In addition, it would be helpful if the strategy 
discussed the importance of implementing parties' efforts to establish 
priorities, milestones, and performance measures that help ensure 
accountability. Ideally, a national strategy would set clear desired results 
and priorities, specific milestones, and outcome-related performance 
measures while giving implementing parties flexibility to pursue and 
achieve those results within a reasonable time frame. If significant 
limitations on performance measures exist, other parts of the strategy 
should address plans to obtain better data or measurements, such as 
national standards or indicators of preparedness. 

Resources, investments, and risk management: This characteristic 
addresses what the strategy will cost, the sources and types of resources 
and investments needed, and where those resources and investments 
should be targeted. Ideally, a strategy would also identify appropriate 
mechanisms to allocate resources. Furthermore, a national strategy should 
elaborate on the risk assessment mentioned earlier and give guidance to 
implementing parties to manage their resources and investments 
accordingly. It should also address the difficult, but critical, issues about 
who pays and how such efforts will be funded and sustained in the future. 
Furthermore, a strategy should include a discussion of the type of 
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resources required, such as budgetary, human capital, information, 
information technology (IT), research and development (R&D), 
procurement of equipment, or contract services. A national strategy 
should also discuss linkages to other resource documents, such as federal 
agency budgets or human capital, IT, R&D, and acquisition strategies. 
Finally, a national strategy should also discuss in greater detail how risk 
management will aid implementing parties in prioritizing and allocating 
resources, including how this approach will create society-wide benefits 
and balance these with the cost to society. Related to this, a national 
strategy should discuss the economic principle of risk-adjusted return on 
resources. 

Organizational roles, responsibilities, and coordination: This 
characteristic addresses what organizations will implement the strategy, 
their roles and responsibilities, and mechanisms for coordinating their 
efforts. It helps to answer the question about who is in charge during times 
of crisis and during all phases of national preparedness: prevention, 
vulnerability reduction, and response and recovery. This characteristic 
entails identifying the specific federal departments, agencies, or offices 
involved, as well as the roles and responsibilities of private sectors. A 
strategy would ideally clarify implementing organizations' relationships in 
terms of leading, supporting, and partnering. In addition, a strategy should 
describe the organizations that will provide the overall framework for 
accountability and oversight. Furthermore, a strategy should also identify 
specific processes for coordination and collaboration between sectors and 
organizations — and address how any conflicts would be resolved. 

Integration and implementation: This characteristic addresses both 
how a national strategy relates to other strategies' goals, objectives, and 
activities (horizontal integration) — and to subordinate levels of 
government and other organizations and their plans to implement the 
strategy (vertical integration). Similarly, related strategies should highlight 
their common or shared goals, subordinate objectives, and activities. In 
addition, a national strategy should address its relationship with relevant 
documents from implementing organizations, such as the strategic plans, 
annual performance plans, or the annual performance reports the 
Government Performance and Results Act requires of federal agencies. A 
strategy should also discuss, as appropriate, various strategies and plans 
produced by the state, local, or private sectors. A strategy also should 
provide guidance such as the development of national standards to link 
together more effectively the roles, responsibilities, and capabilities of the 
implementing parties. 
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U.S. Department of Homeland Security 

Washington, DC 20528 

ilKf Homeland 
IIP Security 



April 21, 2009 

Mr. William O. Jenkins 
Director 

Homeland Security and Justice 
U.S. Government Accountability Office 
441 G Street N.W. 
Washington, DC 20548 

Dear Mr. Jenkins: 

The Department of Homeland Security (DHS) appreciates the opportunity to review and 
respond to the Government Accountability Office's (GAO) draft report entitled, "National 
Preparedness: FEMA Has Made Progress, but Needs to Complete and Integrate Planning, 
Exercise, and Assessment Efforts, " (GAO-09-369). 

The Department appreciates the work done in this engagement to identify any issues that 
hinder the effectiveness of the National Exercise Program. While we may not agree with all 
of the reports assertions, we generally concur with the GAO recommendations, and FEMA 
has already made significant inroads in each aspect of the GAO recommended characteristics 
for sound strategic planning. We believe GAO's recommendations provide a useful 
methodology and sound counsel for revision of our current portfolio of national preparedness 
policy, plans, protocols and procedures. We also observe, however, that much of the report's 
information in areas such as recommending a clear purpose statement, scope, problem 
definition and resources span multiple documents and require a coordinated response. 

There is also a concern that in a number of areas the report suggests that DHS/FEMA should 
hold other federal agencies and departments or state, local or tribal governments, accountable 
for compliance with program requirements. At the same time, the report recognizes in a 
number of places the absence of explicit FEMA authority to compel compliance on the part of 
other federal agencies and departments or state and local governments that are part of the 
highly decentralized, divided and federated homeland security and emergency management 
enterprise that constitutes the National Preparedness System. The report should recognize 
that while improved program management and strategic planning are well-advised and FEMA 
is actively pursuing them, the outcomes still rest on collaboration and cooperation rather than 
mandated compliance. 
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Thank you again for the opportunity to provide comments on this draft report and we look 
forward to working with you again on future homeland security and justice issues. Technical 
comments were previously provided. 



Sincerely, 
Jerald E. Levine 

Director, Departmental GAO/OIQ Liaison Office 
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